"My power is in the people, not in me," he told the crowd.
The rally came after a smaller gathering of about 500 people in his home village on Monday, during which he challenged the interim government to attempt to arrest him.
"We can see that the president does not want to step down voluntarily and instead is issuing calls for actions against the people," Beknazarov said on Tuesday.
"We have opened a criminal case against the former president. If he does not show up today after the rally we will hold an operation to detain him."
Bakiyev was initially guaranteed safe passage out of the country if he stepped down from the presidency, but the self-declared interim government's position appears to have hardened in recent days.
Al Jazeera's Robin Forestier-Walker, reporting from the site of the rally in Jalal'abad, said it was not clear whether the interim government had the forces needed to launch an operation to arrest Bakiyev.
"The president insists if they come to take him, their will be bloodshed. He insists he would not go quietly, he is surrounded by at least a dozen of armed bodyguards," he said.
"The question is how many lives will be lost in such operation, especially he is now in a village surrounded by people would do what they can to support him."
Kyrgyz society is strongly clan-based, but there are few signs that Bakiyev will gain enough support in his southern heartland to return to power.
Meanwhile, the United States is sending a high ranking diplomat to Kyrgyzstan to assess the situation in the country.
Robert Blake, the assistant secretary of state for south and central Asian affairs, said on Monday he would be heading to the region to meet representatives of the de facto government.
Blake said the US has offered support for Kyrgyz efforts "to stabilise their political and economic situation".
He said he had spoken with Roza Otunbayeva, the interim leader, who agreed he should travel there "as soon as possible".
Otunbayeva leads a transitional government that says it will produce a new constitution within six months.
Last week, Otunbayeva reassured Hillary Clinton, the US secretary of state, that the situation would not interfere with an agreement for the US military to use an air base outside Bishkek to ferry supplies into Afghanistan.