"No, no to federalism! No, no to fraud!", chanted the demonstrators, who gathered in the city centre on Friday before heading south to march past the offices of the two main Kurdish parties.
Kurds want Kirkuk to be made the capital of an enlarged autonomous region, and thousands of Kurds who were displaced from the city under Saddam Hussein were allowed to vote two weeks ago.
"There are documents and plenty of evidence showing that fraud took place during the elections in Kirkuk," said a statement which was distributed to protesters and signed by 16 Arab and Turkmen groups.
Among the signatories were the Ankara-funded Iraqi Turkmen Front, the Shia religious party Dawa, and the movement of Shia radical leader Muqtada al-Sadr.
"We ask for new elections to be held in Kirkuk to guarantee they are transparent, because Kirkuk is on the edge of a flaming pit," the document said.
Sunni and Shia Arab parties pulled out of the election in Tamim province around Kirkuk to protest against the authorities' registration of tens of thousands of non-resident Kurds who argued their families had been forced out of the city under Saddam Hussein.
"Kirkuk is on the edge of a flaming pit"
The decision effectively tipped the balance in favour of the Kurds in the city, prompting dire warnings of sectarian violence from Arabs and Turkmen.
A Kurdish weekly reported the main Kurdish alliance was poised to win two-thirds of the vote and take 26 of the 41 seats on Tamim provincial council.