A resident in the area, contacted by Reuters by telephone, said that fighting had begun in Bulobaley, between the towns of Jilib and Buale.
"Fighting has started in Bulobaley... several mortars and rockets have hit the town. I can see many more being fired towards the town," said the resident.
After retreating from Mogadishu, the capital, on Thursday, the UIC had told its fighters to rally around Jilib and the nearby port town of Kismayo.
The UIC forces used bulldozers to dig deep trenches outside Jilib, where analysts believe they have about 3,000 fighters.
As the UIC moves backwards, Kenya has reinforced its northern border and US forces are also said to be in the region.
"The Somali UIC made miracles to bring peace and stability but the world is choosing to go for the so-called government just because of Islamophobia"
Xaqudirir, Mogadishu, Somalia
Send us your views
Early on Sunday, the government and Ethiopian troops, riding in 16 Ethiopian tanks, and accompanied by armoured vehicles and artillery, were 120km north of the front line town of Jilib.
As they headed southwards they were met by as many as 2,000 civilians fleeing the expected fighting.
"The Islamic militia told us they are committed to defend the town to the death, so we have no other option but to flee," Ilse Ali Ilweyn, a father of six who lives in Jilib, said.
Jilib is the first line of defence for the Islamic group and gateway town to Kismayo, the large southern port city 100km further south which is the last major Islamic courts stronghold.
"I want to tell you that the Islamic courts are still alive and ready to fight against the enemy of Allah. We left Mogadishu in order to prevent bloodshed ..."
Sheikh Sharif Sheikh Ahmed A leader of the Islamic Courts Union
The UIC, the umbrella group for the Islamist movement that ruled Mogadishu for six months, has pledged to continue fighting.
"I want to tell you that the Islamic courts are still alive and ready to fight against the enemy of Allah," Sheikh Sharif Sheikh Ahmed, the group's leader, told residents in Kismayo Saturday.
"We left Mogadishu in order to prevent bloodshed in the capital, but that does not mean we lost the holy war against our enemy."
"We decided to fight the enemy in every area," Ahmed told Al Jazeera on Sunday.
Members of the Islamic courts say they still have armed supporters in Mogadishu.
On Saturday night an unexplained blast in the capital killed a woman and wounded two others.
Government takes control
The Somali government - which only two weeks ago was encircled by Islamic fighters in the town of Baidoa - has begun to build up a functioning government in and around Mogadishu.
|Re-building war-ravaged Somalia will be a challenge for any government [AFP]|
Senior members of the government - which is recognised as Somalia's legitimate government by the United Nations and the African Union - began meeting local and tribal leaders in and around Mogadishu on Sunday.
"We should go about re-civilising our country and return to live as we used to in former days," Ali Muhammad Gedi, the prime minister, told Al Jazeera.
The leaders of the Islamic courts said they were willing to negotiate with the Somali government but they said that Ethiopian troops must leave the country.
Somalia has not had a functioning central government since 1991.