Prime minister booed
Felix Kulov, who until December served as Bakiyev's prime minister, told the crowd: "We gathered here today not to seize power by any means but to clean up the authorities."
Almaz Atambayev, who was invited by Bakiyev to become prime minister last week, also addressed the protesters and asked the crowd to trust his government.
"Do not believe the words you are hearing here," Atambayev said to loud boos and jeers, with some demonstrators shouting back "you are wrong".
Ruslan Imankhajiyev, a protest leader, said the opposition planned to set up 100 yurts and 60 army tents to house up to 5,000 activists on the square next to the White House, as the seat of government is known, until their demands were met.
Imankhajiyev said: "We want constitutional and democratic reform. Bakiyev's proposals are useless. We don't trust him."
Officials said about 4,000 police were put on duty to maintain order in Bishkek during opposition rallies.
|Land-locked Kyrgyzstan is home to both Russian and US air bases|
The protests are the latest anti-government demonstrations in the ex-Soviet state that is regarded as the most politically free in Central Asia.
The demonstrators have previously called on the government to tackle corruption and provide more jobs.
"It's impossible to live here, it's impossible to find work. Everyone who can leaves, especially young people," Zhyrgalbek Monolbayev, a 29-year old demonstrator said.
Late on Tuesday, Bakiyev offered constitutional changes curbing his powers as a concession to his opponents.
However, Emil Aliyev, an organiser of the United Front, called this "an attempt to pull the wool over our eyes".
"Look at us: Our eyes are open," Aliyev said.