It was not immediately known if the voters included prisoner Saddam Hussein.
The Independent Electoral Commission in Iraq had said the former leader would be allowed to vote, but its general director, Adel Allami, said on Thursday he did not know whether Saddam was.
The former leader's long-awaited trial is scheduled to begin on Wednesday on charges that he and seven of his regime's henchmen ordered the 1982 massacre of 143 people in a mainly Shia town north of Baghdad following a failed attack on Saddam's life.
In the rest of Iraq, voting would not be held in the constitutional referendum until Saturday.
"The referendum on the draft constitution started early today in all Iraqi and US-run prisons in Baghdad and other provinces," said Husham Al-Suhail, an official with Iraq's Human Rights Ministry.
"Our crews are observing the referendum process at all of the prisons, including Abu Ghraib and Camp Bucca," he said in an interview.
Said Arikat, a spokesman for the UN operation in Iraq, which had dropped off 10,000 copies of the draft constitution at the US prisons, confirmed that voting was under way there.
Officials with the US military declined to comment, referring all questions about voting at its detention centres to the Iraqi government.
More than 12,000 detainees are
being held at Abu Ghraib
More than 12,000 detainees are being held at Abu Ghraib prison, Camp Bucca and two other US military camps in Iraq, many awaiting trial or, in some cases, formal charges.
Many of the detainees are believed to be Sunni Muslim Arabs who were rounded up by US and Iraqi forces on suspicion of supporting Sunni-led anti-government groups.
Detainees also are being held at Iraqi prisons around the country.
The voting began early on Thursday morning at one such Iraqi facility, a prison holding about 260 inmates, including ordinary criminals and suspected armed fighters, in Samawah, a mostly Shia Muslim city about 370km southeast of Baghdad.
There, journalists were allowed to watch detainees as they were searched by police, then lined up in a room of the prison to cast their votes.
"It is a good feeling to be free to vote in the referendum," one prisoner said from behind the bars to an Associated Press Television News crew.
Other prisoners shouted, "Yes, yes, to the constitution."
Kalid Hamza, an Independent Electoral Commission official who was monitoring the voting there, said it started at 7am and that one-third of the inmates had voted by midday.