Many of the refugees have lived in the West African nation for decades.
Mounkaila Modi, Niger's interior minister, told the Associated Press on Tuesday: "We have decided, starting today, to expel these nomadic Arab 'Mohamides' to their home countries.
"These foreigners have shown no respect to the rights of the natives and they're putting pressure on pastures in this region ... we can no longer accept seeing our ecosystem degraded by foreigners."
He said the decision had been taken because many of the Arab refugees, who are mainly nomads known as 'Mohamides', are livestock herders whose camels are destroying arid Niger's fragile ecosystem, particularly water sources which are in short supply.
Many of the Arabs fled past conflicts in Chad decades before.
Modi said that many of the Arab nomads were armed with guns and threatened local inhabitants, though he did not say why.
He said that the order would apply to around 150,000 Arabs in Niger from countries including Algeria, Libya and Chad.
Several residents in eastern Niger told the Associated Press by telephone that security forces there were telling the refugees to leave, but were not yet forcing them to.
There were no reports of any refugees crossing the border.
Modi said the expulsions would take place "with respect to human rights". He gave no other details.