"A young man entered the mosque, shot down the MP and his bodyguard, and then threw a grenade that killed three people and wounded 12," an interior ministry official said.
Saleem al-Jubouri, a spokesman for the Sunni Muslim bloc in parliament, said: "He was in mosque.
"An armed man shot him with a pistol, then threw a grenade at him inside the al-Shawaf mosque."
Police said that the attacker was between 15 and 18-years-old.
"Assassinations of political leaders have a huge effect on national peace, and these acts are meant to stoke renewed sectarian strife between Sunnis and Shias, and also within feuding factions among the Sunnis," Hazim al-Nuaimi, a political analyst at Baghdad University, said.
The attack is the latest to take place before the June 30 deadline for US troops to withdraw from Iraq's town and cities.
The move is part of an agreement between Baghdad and Washington which will see all US combat troops leave Iraq by the end of 2011.
Although violence has dropped in Iraq in recent months attacks are still frequent, particularly in Baghdad and the northern city of Mosul.