The noisy demonstration on Tuesday came as Iraqis celebrated the Muslim Eid al-Adha holiday.
Waving banners and chanting anti-US slogans, about 700 people rallied in the gardens of the Umm al-Qura mosque in the west of the capital to denounce the raid on Saturday night.
"The attack on the Umm al-Qura mosque is an attack on Muslims and Islam," read one of the banners at the protest.
The US military said the raid was linked to the hunt for kidnapped Jill Carroll, a freelance journalist working for the Christian Science Monitor.
Lieutenant-Colonel Barry Johnson, a US military spokesman, said the raid was ordered "as a direct result of a tip by an Iraqi civilian that activities related to the kidnapping were being carried out inside the mosque".
The 28-year-old reporter was seized from a Baghdad neighbourhood by armed men on Saturday.
Her interpreter, Allan Enwiyah, 32, was shot dead and his body abandoned nearby by the kidnappers, while her driver got away.
"Both Iraqi and coalition forces raided the mosque in the early morning hours in order to minimise the impact on worshippers and the surrounding neighbourhood," Johnson said.
Six people were detained for questioning, he added.
Carroll was seized from Baghdad
by armed men on Saturday
The Association of Muslim Scholars (AMS) which is based at the mosque, confirmed that one of its members, Yunis Aikali, and five mosque guards were arrested in the raid.
In a statement, AMS also accused US soldiers of desecrating the mosque and carrying away files containing the names of members.
"We call on the occupiers to withdraw from Iraq because they are the reason for every crime and the death of every innocent in Iraq," Harith al-Aubaidi, a member of AMS said in his sermon at the prayers ahead of the protest.
Ashraf Qazi, the UN special representative in Iraq, also deplored the raid.
"This again underlines the importance of all parties respecting the sanctity of holy sites and places of worship," he said in a statement.
Killed in the act
Also on Monday, two armed men were killed in the Iraqi city of Sammara, north of the capital, when a bomb they were trying to plant at the side of a road exploded prematurely.
Elsewhere, US soldiers shot dead one armed man and arrested another after they had opened fire on a US patrol.
"We call on the occupiers to withdraw from Iraq because they are the reason for every crime and the death of every innocent in Iraq".
Separately, US soldiers killed an armed man after he shot at them from a building near Balad, north of Baghdad. They said they found bomb-making equipment in the building.
Iraqi security forces, meanwhile, were on alert in western Baghdad looking for hostages, security sources said.
The US embassy said it had nothing new to report on Carroll.
The Christian Science Monitor said it was urgently seeking information about its reporter after confirming her abduction on Monday.
Carroll's driver, quoted in a story posted on the Monitor's website, said armed assailants jumped in front of the car, pulled him away, and drove off with their two captives all within 15 seconds.
Several Westerners are currently being held hostage by Iraqi fighters, including an American, a Briton and two Canadians who are members of a Christian peace group.
Carroll was the 31st media worker to have been kidnapped in Iraq since the start of the war in 2003, according to watchdog group Reporters without Borders.
Five of the kidnapped journalists - four Iraqis and Enzo Baldoni of Italy - were killed by their abductors. The others were released.