The 53-nation body condemned the 3 August coup and suspended Mauritania's membership of the organisation, but has stopped short of calling for exiled President Maaouiya Sid'Ahmed Ould Taya to be restored to office.
Taya, who had ruled since a 1984 coup, was widely unpopular and most Mauritanians welcomed his ousting.
The African Union (AU) delegation includes Nigerian Foreign Minister Oluyemi Adeniji, South African Safety and Security Minister Charles Nqakula and an official of the African Union Commission.
They are expected to meet the newly declared president, Colonel Ely Ould Mohamed Vall, and other leaders of last week's coup.
Nigeria holds the chairmanship of the AU, and South Africa is this month's head of the organisation's Peace and Security Council.
Speaking in an interview on Monday broadcast on the Arabic news channel Al-Arabiya, Taya vowed he would return to power and called on his country's armed forces to reverse the coup.
Taya issued orders "in my capacity as president of the republic to the armed forces to restore the natural order and put an end to this crime".
"I am determined to return to Nouakchott to continue the job of building our nation," he added.
Taya on Tuesday left the west African state of Niger where he had been given refuge and headed for The Gambia, an aide of Niger President Mamadou Tandja said.
Taya, who was toppled by top army brass while he was abroad on Wednesday, left for Banjul, the capital of the tiny West African state, he added.
The Gambia, a narrow strip of land that is surrounded on three sides by Senegal, is an English-speaking state that does not have a border with Mauritania.
The bloodless military coup was welcomed by many who had grown weary of Taya's harsh 21-year rule.
Taya urged the AU to keep up pressure on the new rulers.
President Maaouiya Sid' Ahmed
Ould Taya vowed to return
The AU said on Monday that it was dispatching the delegation to Nouakchott "to discuss with the new Mauritanian authorities ways of restoring constitutional order".
On Monday, the US State Department said it backed the trip but stopped short of calling for Taya's return.
Vall has pledged to hold elections within two years and says no member of the military that seized power will stand in the vote.
Taya was criticised by most Mauritanians for allying himself with the United States in the "war on terror" and opening full diplomatic relations with Israel six years ago, becoming one of three Arab League nations to do so.
Taya's crackdown on opponents is believed to have contributed to bringing on the coup, but a power struggle over recently discovered offshore oil reserves may have played a role.