"While agriculture recovery is indeed vital, forcing people home without aid makes it harder for aid agencies to reach them with assistance," said a statement from the group.
A UN official also said on Friday the government was making cyclone survivors leave the camps and "dumping" them near their devastated villages with virtually no aid supplies.
Eight camps set up for homeless survivors in the Irrawaddy delta town of Bogalay were "totally empty" as authorities continued to move people out of them, Teh Tai Ring, of the United Nations Children's Fund (Unicef), said.
His comments were made at a meeting of UN and private aid agency workers discussing water and sanitation issues.
He said: "The government is moving people unannounced."
He also said the authorities have been "dumping people in the approximate location of the villages, basically with nothing".
After his statement was reported, Unicef issued a statement saying the remarks referred to "unconfirmed reports by relief workers on the relocation of displaced people affected by"the May 2-3 storm".
However, Teh said the information came from a relief worker who had just returned from the affected area and that "tears were shed" when he recounted his findings earlier in the day.
'Despair and pain'
More than 400 cyclone victims from a delta township of Labutta were evicted from a church in Yangon on Friday following orders from authorities a day earlier.
"It was a scene of sadness, despair and pain," said a church official at the Yangon Karen Baptist Home Missions.
"Those villagers lost their homes, their family members and the whole village was washed away. They have no home to go back to."
All the refuge seekers except some pregnant women, two young children and those with severe illnesses left the church in eleven trucks on Friday morning.
The authorities told church workers that the refugees would first be taken to a government camp in Myaung Mya - a mostly undamaged town in the Irrawaddy delta - but it was not immediately clear when they would be resettled in their villages.
Aid groups continue to accuse Myanmar's military government of hindering foreign assistance for victims of the cyclone, despite a promise to Ban Ki-moon, the UN secretary-general, to ease travel restrictions.
Some foreign aid workers are still awaiting visas, and the government is taking 48 hours to process requests to enter the Irrawaddy delta, the groups said.
The International Red Cross is reportedly still waiting for permission to send 30 foreign workers into the delta.