The minister of minerals and water, Mohamud Salad Nur, and three assistant ministers announced their resignation in Baidoa, the provincial seat of the interim government of Abdullahi Yusuf, the president.
Sayeed Hassan Shire, one of the assistant ministers who resigned, said: "This is our decision because this cabinet has failed the reason for which it was established. It has failed to reconcile the Somali people."
Eighteen other officials resigned from the 102-member parliament last week.
The prime minister, Ali Mohamed Gedi, narrowly survived a confidence vote on Sunday.
The government has called for peace talks with Islamists to be postponed, Gedi said on Tuesday.
"I have communicated with the Arab League and requested that they postpone the talks for 15 days. They have accepted my request," he told reporters.
The Islamists have increased their control in central Somalia since local militiamen gave up their weapons and vowed to support them.
Weapons handed over
Militiamen from the Hawiye clan handed over at least 50 technicals - pickups mounted with machineguns - to the Supreme Islamic Council of Somalia (SICS) in Adado township in the central Galgudud region.
Sheikh Hassan Dahir Aweys, head of the SICS, said: "We came here by the wishes of the locals not by force.
The cargo aircraft landed in
Mogadishu twice last week
"Our main objective is to establish an Islamic court here in central regions."
Local leaders said they would back the Islamists whose growing influence has threatened the authority of the transitional government.
The group has also opened a new sharia court in the north of the country.
Mohamed Qoryarey, leader of Islamic Courts' militia in Adaado district, said: "If the residents of every area in Somalia ask us to ... we should go and install Islamic courts."
Authorities in Kazakhstan are investigating reports that a aeroplane with the insignia of the former Soviet republic delivered weapons for the Islamist militia, an official said on Tuesday.
An Ilyushin-76 cargo aircraft, with the Kazakh flag painted on its tail, landed in Mogadishu on Wednesday and Friday of last week, witnesses said. The Somali government said it was delivering weapons from Eritrea.
The Islamists and Eritrean officials have denied the accusation.