He also said he would sell a plane that Ravalomanana recently bought for $60m, and use the money "to establish a hospital for the people's health, which is a higher priority".
Rajoelina also cancelled an agreement to lease South Korean corporation Daewoo more than a million hectares of land in Madagascar to grow food crops, a deal for which Ravalomanana was heavily criticised.
After months of power struggle, Ravalomanana handed over presidential powers to the military on Tuesday and the military, in turn, handed power to Rajoelina.
The fact that the army did not take over the leadership of the country means the African Union may not brand it a coup, which would have meant suspending Madagascar from the grouping.
But South Africa and Zambia have denounced Rajoelina's takeover.
"[Zambia] calls for the immediate suspension of the membership of Madagascar in SADC [Southern African Development Community] and the African Union for the return to constitutional order within the shortest possible period," Kabinga Pande, the foreign minister, said.
The SADC is scheduled to meet in Swaziland on Thursday to discuss how to handle the situation.
And France has criticised Rajoelina's declared intention to hold elections in two years, with Eric Chevallier, a foreign ministry spokesman, saying "the 24-month period to organise fresh elections is too long".
"Under such extraordinary circumstances, the international community wants a quick return to democracy," he said.
Ravalomanana has not been seen in public since he resigned and Rajoelina said on Wednesday that he still wanted to see the deposed president arrested.
Despite the mass of cheering supporters over the past two days in the capital, Stephen Ellis, a historian, told Al Jazeera that Rajoelina may face some hurdles in winning the support of the country as a whole.
"[His claim to power] is clearly unconstitutional and there is going to be a period of instability," he said of the former mayor of the capital.
"Now we are in for quite a rough ride because the new government is going to be illegitimate."
At least 135 people have been killed since the country's political crisis began in January, most of them when security forces cracked down on anti-government protests at the order of Ravalomanana's government.