Imam Mahdi Ahmed Saleh al-Sumaydah was detained along with more than a dozen worshippers and staff of the Ibn Taimiya mosque in the Yarmuk district of the capital, they said.
The US military was unable to immediately confirm the operation, said to have begun at around 7:30 am (04:30 GMT), but under normal practices they do not enter mosques unless under fire or accompanied by Iraqi liaisons.
"American tanks and armoured personnel carriers forced their way through the front gate of the mosque, while helicopters flew overhead," said one worshipper.
He said troops damaged property after they entered the 35-year-old holy building.
"They went directly into the mosque, violating its sanctity," he added.
"The Americans are wrong. All the people connected to this mosque are for the unification of Sunnis and Shias"
Although the mangled front gate of the mosque appeared to have been freshly torn from its hinges, it was not possible to verify claims of internal damage to the building.
Other worshippers said the raid had taken place because Sumaydah was wrongly suspected of fuelling insurgency in Iraq linked to tensions between Shia Muslims and minority Sunnis, from whose ranks former president Saddam Hussein hails.
"The Americans are wrong. All the people connected to this mosque are for the unification of Sunnis and Shias," one said.
In northern Iraq a US military helicopter was forced to land on Thursday after an apparent mechanical failure and some soldiers may have been injured, a US military spokesman said.
A spokesman for the 101st Airborne Division, which controls a broad swathe of northern Iraq, said the aircraft had come down at about 15:30 (12:30 GMT), but played down the possibility of it having been hit by hostile fire from the ground.
Insurgents downed three US
helicopters last November
"It was potentially a bit more than a precautionary landing, perhaps a hard landing with some injuries...most likely it was a technical failure," he said.
Asked if the helicopter had come under fire, he said: "There is no suggestion (of that) at this time."
Two US helicopters made similar landings last month, and guerrillas last November shot down three US helicopters, killing 39 US soldiers.