The interim government of Kyrgyzstan has struggled to impose its authority in the south since assuming power after a revolt toppled the administration of president Kurmanbek Bakiyev in April.
Bakiyev, who is in exile in Belarus, retains strong political support in the south of Kyrgyzstan, which is also his family's stronghold.
Myrzakmatov's show of force on Friday challenged the authority of the interim government.
But Azimbek Beknazarov, a deputy head of the interim government, also appeared at the Osh rally in an attempt to reassure residents that Myrzakmatov was not dismissed.
Beknazarov told the protesters that Myrzakmatov was still the mayor of Osh, in spite of being offered a position in the interim government.
"They offered Melis many different positions but he refused," he said.
Earlier, Farid Niyazov, a spokesman for Kyrgyzstan's interim government, also said that no order was ever issued to sack the mayor.
"Protesters are demanding the mayor be restored to office, although there has been no such order to remove him from the post," Niyazov was quoted as saying by the state news agency.
Friday's rally came ahead of the country's general elections scheduled to be held on October 10.
"If the demands of the Kyrgyz people and the votes of the southern capital Osh are not heard, we will boycott the parliament [and] the parliamentary elections, which will be held this autumn," Toktosh Alunbekov, a protester in Osh, said.
Al Jazeera's Robin Forestier-Walker, reporting from neighbouring Kazakhstan, said that the government in Kyrgyzstan faces many challenges to keep the divided country together ahead of the upcoming elections.
"Not least of which is this mayor in Osh, who is the only surviving official from the Bakiyev regime.
"And among the ethnic Uzbek community, many believe he may have played a hand in the violence that took place in June," our correspondent said, referring to intense clashes that erupted in the south between ethnic Kyrgyz and Uzbeks in June in which nearly 400 people were killed.
"The mayor himself denies this, but he has been standing firm amid rumours that the government would try to remove him."