At least one faction of the army has revolted and the rest of the military has refused to act against opposition protesters.
Madagascar's army has remained traditionally neutral during periods of political volatility since independence from France in 1960.
Rajoelina accuses Ravalomanana of being a dictator and has tapped into widespread public discontent, especially among Madagascar's poor.
"The country cannot afford to wait under such difficult circumstances"
Col. Andre Ndriarijaona,
leader of army mutiny
More than 130 people have been killed in Madagascar since the country's political crisis began in January, most of them when security forces cracked down on anti-government protests at the order of Ravalomanana's government.
Ravalomanana has offered to hold a referendum to end the crisis, but the Rajoelina said in a radio address broadcast that there was no need for a poll as the people had already made their opinions clear.
"We are not interested in this referendum," he said.
Colonel Andre Ndriarijaona, who led an army mutiny and ousted the army chief, told the Reuters news agency that a referendum would take too long.
"The country cannot afford to wait under such difficult circumstances. Perhaps it offers a democratic solution, but my worry is the chaos in the meantime," he said.
Explosions were heard near the presidential palace early on Monday, raising concerns of possible violence.
Officials reported two or three blasts close to the palace, where some of Ravalomanana's supporters are gathered, but there was no indication that the blasts represented an attack.
A colonel with the presidential guard said that three shells were fired, landing on a bypass near the palace.
"This was to intimidate the mass of supporters," he said. Local residents confirmed that there had been explosions.
Rajoelina, a former disc jockey who was sacked as Antananarivo's mayor earlier this year, says Ravalomanana is an autocrat running the island like a private company.
Ravalomanana's supporters call Rajoelina a maverick and troublemaker bent on seizing power illegally.
The UN has sent Tiebile Drame, Mali's former foreign minister, to mediate on its behalf.
"The way out of this crisis lies with the Malagasy people," he told the Reuters news agency.
"Madagascar has to turn its back on the cycle of violence. The only possible way of reaching a solution is through dialogue and democracy."