The group had seized land in the village of Mayevka outside Bishkek and demanded talks with Bishkek officials, local media reported.
"The provisional government has worked out a democratic development plan tentatively dubbed 'The return to democracy'"
Omurbek Tekebayev, interim prime minister
"Kyrgyz from different regions of Kyrgyzstan are setting houses on fire and taking land from Turks and Russians," Viktor, an ethnic Russian resident of Mayevka, was quoted by the AFP news agency as saying.
"This is a real war."
After moving into the capital, the mob forced the acting mayor of Bishkek to sign a document authorising the handover of the land they had seized.
The interim government later responded with a show of force, including hundreds of police officers.
"Around 600 police supported by armoured personnel carriers have been sent on patrol in the village of Mayevka," a source in the Kyrgyz interior ministry told the AFP news agency.
Kyrgyzstan has remained tense since Bakiyev was ousted in the aftermath of the violent unrest that forced the president from power and left 84 people dead.
About 1,500 Bakiyev loyalists gathered in the city of Jalal'abad on Monday to protest against the interim government.
"Bakiyev is our legitimate president!" they shouted, some holding banners reading "The opposition spilt blood to grab power".
Others gave out leaflets calling for Bakiyev's return.
Supporters of Bakiyev, who was flown to Kazakhstan amid concerns about his safety on Thursday, seized a regional government office over the weekend.
A Kazakh foreign ministry spokesman said on Monday that Bakiyev had left Kazakhstan but did not say where he had gone.
In a bid to claim its authority over Kyrgyzstan, the interim government unveiled a reform plan on Monday, which includes shifting powers from the president to parliament and holding free parliamentary and presidential elections in late September or early October.
"The provisional government has worked out a democratic development plan tentatively dubbed 'The return to democracy'," Omurbek Tekebayev, the interim prime minister, said.
He said UN officials would be invited to join the Central Election Commission in order to maximise transparency.
The interim government has cracked down on Bakiyev's allies, issuing arrest warrants for officials it says were involved in murders and corrupt deals under Bakiyev.
Bakiyev himself came to power in a 2005 revolt on the back of promises to build democracy and protect human rights. He was then backed by Roza Otunbayeva who now heads the interim government.