Madagascar's army had traditionally remained neutral during political volatility, but Colonel Andre Ndriarijaona, the head of the armed forces, said on Monday the military was 99 per cent behind the opposition.
"We are there for the Malagasy people. If Andry Rajoelina can resolve the problem, we are behind him," he said.
Ravalomanana is currently in another presidential palace, known as Iavoloha, a few kilometres from the centre of Antananarivo, the country's main city.
Al Jazeera's Haru Mutasa, reporting from Antananarivo, said there was a lot of military deployed on the streets as people woke up on Tuesday morning.
The army had vowed to take the second presidential palace during the night, but no such action had yet been taken and it appeared that Ravalomanana was planning to remain at the palace, she said.
Amid the developments, Rajoelina prepared to gather his supporters.
"Rajoelina is going to hold a rally in the centre Antananarivo and it's then thought he will tell his supporters to follow him on foot towards the administrative offices where he will open the door and walk in, as a a symbolic show that he has taken over," our correspondent said.
Rajoelina, a 34-year-old former disc jockey, was dismissed last month by Ravalomanana's government as the mayor of Antananarivo.
Rajoelina accuses Ravalomanana of being a dictator and misusing public funds.
In turn, Ravalomanana's supporters call Rajoelina a troublemaker bent on seizing power illegally.
Under growing pressure to resign, Ravalomanana offered on Monday to hold a referendum to let the people decide who should run Madagascar.
But Rajoelina rejected the offer, saying the people had already made their opinions clear.
Even as he called on the security forces to arrest Ravalomanana, Rajoelina denied ordering Monday's army action, saying: "It wasn't me who gave the order [to take the palace]."
"Do we intend to seize Iavoloha?" Rajoelina said, referring to the palace where Ravalomanana currently is.
"Many things will take place in the next 48 hours. Right now Ravalomanana has no power".
Our correspondent said that while Rajoelina enjoyed strong support in Antananarivo because he was formerly the city's mayor, it remained unclear if he could muster the backing of people and troops in the rest of the country.
At the regional level, the African Union has called the Madagascan military's intervention an attempted coup and urged the people to respect the constitution.
"The situation in Madagascar is an internal conflict," Edouard Alo-Glele, Benin's envoy to Ethiopia, said after an emergency meeting of the AU's Peace and Security Council.
"It is an attempted coup d'etat. We condemn the attempted coup d'etat."
At least 135 people have been killed since the country's political crisis began in January, most of them when security forces cracked down on anti-government protests at the order of Ravalomanana's government.
The crisis, with Rajoelina leading anti-government demonstrations since the start of the year, has also damaged the country's nearly $400m-a-year tourism sector.