A witness said that about 20 men carrying sub-machine guns, anti-tank rockets and grenades emerged on Wednesday morning from a funeral procession for the worshippers killed in the mosque blast.
The building was once used by the ousted Baath party and one floor has been converted into a Shia mosque named al-Tawhid in the al-Dabash mixed area of northwest Baghdad.
“They came to me for the key because the shaikh was not there. But as I didn't have it, they broke down the door, fired shots in the ceiling, tore up the posters of Imam Hussein and other martyrs and burnt our religious books," said Muhammad Abd.
Another witness said the men broke down the mosque’s door and burnt books.
Three armoured vehicles and US soldiers were posted outside the Shia mosque on Wednesday afternoon.
A US sergeant, who refused to give his full name, said they were protecting the building after an attack.
The day before a booby-trapped bomb exploded in the courtyard of the Akhbar al-Mustafa mosque, said police.
Abd al-Aziz Hakim, rotating chairman of Iraq's US-appointed Interim Governing Council and a Shia leader, condemned both attacks.
“They are part of a confessional scheme followed by those who carried out these two crimes," he said, urging police to find the attackers quickly.
There have been increasing calls from clerics across the country's religious spectrum to avoid stoking tensions between sects in the war-torn country.