Each of the three main islands in the Indian Ocean archipelago has a regional president.
Abdul Rahim Said Bakar, a Comoros government spokesman, told Al Jazeera that troops have not yet found Colonel Mahamed Bacar.
"We have noticed there is a pocket of resistance around from his house, not far from Ouani, so we suppose that he is on his own at this moment. The next step for us is to capture him," Bakar said.
"We need to capture him, take him to court so he can answer his crimes against humanity and the people of Anjouan."
The federal government has refused to recognise Bacar's re-election as the island's president last June and vowed to remove him.
Weapons fire from high-calibre machine guns began at 5am local time (0200GMT) and there were occasional explosions, according to AFP news agency.
There was no independent confirmation of the government's claim from Anjouan, where phone lines have been cut.
The Indian archipelago nation consists of three islands: Grand Comore, Moheli and Anjouan
Each island has its own president and administrative authorities
The Union of Comoros central government is led by President Ahmed Abdallah Sambi
Anjouan is home to about 300,000 of the archipelago's 700,000 population
The country has seen 19 coups or attempted coups since independence from France in 1975
Mohamed Dosara, defence chief of staff troops for the federal government, said: "For the moment, I don't have the correct information about his whereabouts, but perhaps by the end of the day we will get some news on that."
He said troops faced a small amount of resistance but he did not give any details of casualities.
Ahmed Abdallah Mohamed Sambi, the president of Comoros, on Sunday announced that he had given the green light to a long-threatened joint operation by Comoran and AU forces to reunify the Comoros.
An AU-backed invasion force set off from the Comoros island of Moheli for Anjouan on Monday.
About 1,500 AU troops, mostly from Tanzania and Sudan, and about 400 Comoran soldiers are involved in the military operation to remove Bacar.
Bacar is believed to be in command of an estimated 300 troops.
The Comoros' main leadership is based in Moroni, on the main island of Grand Comore.
"I have ordered the Comoran army and the the forces of our country's friends to bring Anjouan back under the rule of law and free her citizens," Sambi said on Monday.
He said he did so "without joy, like swallowing a bitter pill".
The central government used helicopters to drop leaflets on Anjouan before the assault, telling civilians to stay in their homes.
The National Development Army "informs all the residents of Anjouan that it will be in Anjouan within days or in the coming hours," the leaflets said.
Idi Nadhoim, the Comoran vice-president, said before the operation: "Colonel Bacar will be arrested if he does not flee and will face Comoran courts for treason, usurpation of power, torture and war crimes, as documented by complaints which Anjouanese victims have filed."
On Sunday, the AU troops sent to the Indian Ocean island nation practised beach landings for the operation, named Democracy in Comoros.
Comoros has experienced 19 coups or coup attempts since it acquired independence from France in 1975.