Aljazeera television reported that it had received a statement from a group calling itself the Islamic Army of Iraq, claiming responsibility for the diplomat's assassination on Wednesday in the Azamiya district of the capital.
Qussay Mehdi was reportedly accosted by four armed men who tried to kidnap him. He put up a fight and when the armed men realised they could not take him, they fired twice.
A source said one shot hit Mehdi in the head.
Mehdi, Iraq's charge d'affaires to the United Arab Emirates from June 2003 until last month, had been working at the interim foreign ministry awaiting his next assignment.
A seasoned diplomat in his mid-to-late 50s, Mehdi had also worked at the foreign ministry during Saddam Hussein's rule.
His family, who still live in the UAE, have been informed about his death and are returning to Iraq, a foreign ministry source said.