"They [al-Shabab] launched their offensive on several fronts very early this morning. The fighting is very intense but we are holding up," he said.
Hundreds of families fled the area, many of the leaving ahead of the fighting as the two groups geared up to battle for control.
Kismayo, about 300km from the capital, Mogadishu, has been relatively quiet since it was wrested from interim Somalia government forces by Islamist fighters in August.
But relations between the two groups controlling it have soured in recent weeks.
The two factions had agreed to share power in Kismayo, with each governing for six months alternatively. But clan politics reportedly caused the rotation to fail when al-Shabab refused to relinquish the administration.
Kismayo had attracted many Somalis who had fled Mogadishu, which has been plagued by almost uninterrupted violence over the past three years.
The trouble in Kismayo had been brewing for weeks and Hizbul Islam leaders earlier threatened to fight al-Shabab "everywhere" in Somalia if clashes began at the port, a lucrative source of taxes and other income.
Western donors have long hoped that extremists in al-Shabab could be isolated by a deal with more moderate Hizbul Islam leaders.
Meanwhile, security analysts warn Somalia has become a safe haven for fighters, including foreign jihadists, and Washington accuses al-Shabab of being al-Qaeda's proxy in the country.
Fighting in Somalia has killed nearly 19,000 civilians since the start of 2007 and driven 1.5 million from their homes.