The donkeys, bought in neighbouring Tanzania, were given to residents of a village in Gitega province as part of a project by a local NGO to help women and children transport agricultural products, water or wood.
However, a presidential adviser described the project as “an insult to the nation”.
Gabby Bugaga, spokesman for the Senate president, also wrote on Twitter the French were “taking us for donkeys”.
“Be honest, is the donkey a symbol of a quality or a flaw,” he wrote.
Donkeys are not indigenous to Burundi.
On Sunday, Agriculture Minister Deo Guide Rurema asked a local administrator to “facilitate the immediate withdrawal of all donkeys that have been distributed … without respecting the technical procedure of the distribution of exotic animals”.
Last Thursday, the day the project was inaugurated, French Ambassador Laurent Delahousse praised “the introduction of the Land Cruiser of the animal kingdom to Burundi”.
After the controversy, Delahousse said to his knowledge “all procedures were respected”.
A European diplomat, speaking on condition of anonymity, told AFP news agency Burundi was hitting back at France over a statement it made criticising a referendum earlier this month. The vote reformed the constitution allowing President Pierre Nkurunziza to seek another two terms in office so he can potentially remain in power until 2034.
The diplomat said a similar project involving donkeys in Ruyigi province, financed by Belgium, had not been met with any problems.
Burundi was plunged into crisis in 2015 when Nkurunziza sought a third term in office, sparking protests and violence that killed at least 1,200 people and displaced 400,000 others.
The International Criminal Court has said it is investigating alleged state-sponsored crimes against humanity in the country.
In 2017, Burundi became the first nation to leave The Hague-based court.