The arrest on Sunday was the latest move in a campaign by Azeri authorities to evict worshippers from the Juma mosque in the capital Baku.
Human rights groups say the campaign violates international conventions on freedom of worship.
As worshippers began evening prayers in the mosque, a senior policeman entered the building and ordered them to leave.
When they continued to pray, several men in civilian clothes - following orders from the officer - grabbed the cleric leading the prayers, Adil Husseinov, carried him outside and bundled him into a police car.
Police then ordered the remaining worshippers to leave the mosque. Witnesses said that two worshippers were also arrested outside the mosque.
In Islamic tradition, it is considered sacrilegious to interrupt someone once they have begun their prayer ritual.
"This is an atrocity," said Seimur Rashidov, a religious freedoms campaigner who was at the mosque. "It is not just blasphemy, it is an insult as well."
The controversy surrounding the mosque has attracted the attention of Western governments and human rights bodies, which are already concerned about the former Soviet republic's worsening human rights record.
The US State Department said last week it was following the issue "with concern."
Government officials argue that the Juma mosque religious community is illegally occupying the building, following a court decision in April which ruled that it did not have a valid rental agreement.
The mosque has been denied a legal status because it refuses to register with the state-sanctioned religious authorities.
Azeri riot police beat a protestor
shouting anti-government slogans
Authorities keep close tabs on religious groups in Azerbaijan, a secular state where most of the eight million population are Shia Muslims.
But critics of Azerbaijan's government say the Juma mosque is being targeted because Ilgar Ibrahimoglu, its imam, or chief cleric, is a leading human rights advocate and critic of the authorities.
Last week, court-appointed bailiffs evicted the Juma mosque community from its offices inside the mosque compound. The religious authorities also appointed a new imam to take charge.
But worshippers have refused to recognise the new imam and have continued to pray in the mosque with their own clerics, in defiance of the authorities.