The military council now running the country also pledged to respect treaties and other international commitments binding Mauritania, Africa's newest oil producer.
Military convoys rolled through the capital on Wednesday, surrounding the presidential palace, the prime minister's office and the state broadcaster.
"There is no sight of any military presence in the street except in front of the presidential palace and the radio station," Mohamed Vall, Al Jazeera's correspondent in Nouakchott, the Mauritanian capital, said.
"There isn't even a curfew here. This situation is calm, people are going about their usual business."
Vall also reported that a rally in support of the coup was due in Nouakchott for later on Thursday.
Officers seized Abdallahi, along with the country's interior minister and Yahya Ould Ahmed Waghf, the prime minister, in a bloodless coup led by General Mohamed Ould Abdel Aziz.
Abdel Aziz is the leader of the presidential guard who was fired by Abdallahi shortly beforehand.
Abdallahi himself won elections last year after a 2005 coup, also instigated by Ould Abdel Aziz, which ended years of military rule.
The coup triggered international condemnation, with the US urging the release of Mauritania's leaders and the EU threatening to cut off aid.
Condoleezza Rice, the US secretary of state, called on the military to release the president and prime minister "and to restore the legitimate, constitutional, democratically elected government immediately".
Ban Ki-moon, the UN secretary-general, said he "deeply regrets the overthrow of the government of President Sidi Ould Cheikh Abdallahi," and called for "the restoration of constitutional order", his spokeswoman said.
The African Union called for maintaining "constitutional legality" and said Ramtane Lamamra, its peace and security commissioner, would go to Mauritania.
Condemnation also came from regional powerhouses South Africa and Nigeria.
Mauritania has been facing a political crisis for months, with 48 members of parliament walking out on the ruling party less than two weeks after a vote of no confidence in the government prompted a cabinet reshuffle.