The ethnic Hazara backers of former Planning Minister Haji Muhammad Muhaqiq marched through the dusty streets of the northern town of Mazar-i-Sharif to demand his reinstatement.
"Karzai should leave the office!" they shouted. "He is not a national leader. We want Mohaqiq reappointed."
Government officials say Muhaqiq resigned from the cabinet at a stormy meeting on Sunday.
But he says he was fired after announcing his intention to run against Karzai in presidential polls due to be held this year and for protesting against Karzai's decision to transfer some of his ministry's powers to Finance Minster Ashraf Ghani Ahmadzai.
First to leave
Muhaqiq, a former commander of the Northern Alliance, was the first minister to leave Karzai's cabinet since the US-backed government was installed in late 2001 after the overthrow of the Taliban government.
His ethnic Hazara troops have been accused of committing atrocities against factional rivals during civil warfare in the 1990s.
"Karzai should leave the office!"
supporters of Muhaqiq
Mazar, about 300km northwest of Kabul, is Muhaqiq's main support base.
His departure from the cabinet came as Karzai attempts to reduce the influence of the powerful Northern Alliance in government and install more professional ministers.
Alliance commanders secured many key government posts after helping US-led forces overthrow the Taliban in late 2001.
Muhaqiq was replaced in the key planning post by Ramazan Bashardoost, a veteran Western-educated technocrat.
Karzai is widely seen as a favourite to win what will be Afghanistan's first democratic elections. The polls are due to be held in June, but are expected to be delayed due to the slow pace of voter registration.