Witnesses said fighters from the Islamic Courts Union were in the town of Bur Haqaba, 60km from the government's base of Baidoa.
A militia official said that his forces will soon seize the base of the internationally recognised transitional government.
Sheikh Muqtar Robow said: "Nothing will stop us from going in to Baidoa."
He said that more than 130 fighters who had been loyal to Abdullahi Yusuf, the Somali president, had defected to the Islamists.
The government is on high alert and ready to defend itself from an attack, the deputy information minister, Salad Ali Jelle, said.
Ali Mohamed Gedi, the Somali prime minister, said "several" Islamist fighters with around 30 battlewagons - machine gun-mounted pickups also known as "technicals" - were positioned southeast of Baidoa.
Gedi urged the Islamists to send their fighters back to Mogadishu and allow peace talks to go ahead at the weekend.
"I appeal to them to go back to Mogadishu, stop attacking other parts of Somalia and needlessly displacing civilians," he said.
The Islamist group's deputy defence chief was contradicted by a senior cleric from the Supreme Islamic Council of Somalia in Mogadishu who said that there were no plans for the fighters to carry on to Baidoa.
Ethiopia's government spokesman said the country is prepared to invade Somalia to defend the government.
Up to 2,000 Ethiopian troops crossed the border last week with several tanks to join about 2,000 soldiers already there, various sources have claimed.
Relations between the government and Islamists have been tense since the Islamic courts took control of Mogadishu last month, challenging the authority of the largely powerless government.
The two sides agreed a truce and mutual recognition deal in Sudan on June 22 – the government says the Islamists have broken the deal.
They were due to hold further Arab League-sponsored talks in Khartoum last weekend but the government boycotted them. On Monday, officials changed their minds and the talks were rescheduled for this Saturday.
Somalia has been without effective central government since clan-based warlords overthrew the president, Mohamed Siad Barre, in 1991 and then turned on each other.