"After deep reflection, I have decided to dissolve the government and give up power so that a military directorate can be established,'' he said in a radio address.
"This decision was very difficult and very hard, but it had to be made. We need calm and peace to develop our country."
'Rough road ahead'
Rajoelina had already declared himself head of the transitional government and taken an oath of office after entering presidential offices in the centre of the capital, Antananarivo, earlier on Tuesday.
"I solemnly declare that I will not spare any effort," he said, proclaiming that the transitional authority he set up last month was in charge of the country's affairs.
"We are now free but the road ahead remains rough."
Thousands of supporters cheered as he travelled to the compound, which had been seized by troops on Monday.
"This is a David versus Goliath victory," one excited opposition supporter yelled.
Rajoelina had repeatedly attacked Ravalomanana, accusing him of misspending public funds and undermining democracy in Madagascar.
The African Union had warned Madagascar's military against handing over power to Rajoelina.
"If the military hand over power to the mayor, it is not constitutional," Jean Ping, the African Union commission chairman, said before the decision was announced.
Ping said that any taking of power by non-constitutional means would be considered a coup d'etat by the body. According to the AU's charter, coups or unconstitutional changes of government are cause for automatic suspension.
Bruno Nongoma Zidouemba, Burkina Faso's ambassador to the 53-state AU, urged Rajoelina and the military to ensure Ravalomanana's safety.
|Ravalomanana's whereabouts were unclear after he waved goodbye to office [AFP]
"We have launched an appeal for law and order to prevail in Madagascar, we have also called for president Ravalomanana's security to be guaranteed, as well as that of his relatives and his entourage," he said.
The president's spokesman said that Ravalomanana, who had been sheltering at a presidential palace about 6km outside the capital, had been moved to an undisclosed location.
Robert Wood, a US state department spokesman, dismissed reports circulating in Madagascar that Ravalomanana had taken refuge in the US embassy in Antananarivo.
At least 135 people have been killed since the country's political crisis began in January, most of them when security forces cracked down on anti-government protests at the order of Ravalomanana's government.