The UN's World Food Programme (WFP) deputy director in Kampala on Saturday said local villagers, as well as refugees from neighbouring Sudan, had been targeted.
Edward Kallon added there were an increasing number of attacks by the Ugandan rebels over the last month.
He estimates between 20,000 and 25,000 desperate people are now on the move.
Refugees twice over
"It is very, very unusual for Adjumani. This is one of the areas where the Sudanese refugees are being hosted, and now we have the refugees being displaced alongside the host population. It is a very complex situation."
Kallon said rebels based in lawless areas of southern Sudan had broken up into small bands and were increasingly attacking Sudanese refugee camps in outlying areas of northern Uganda.
"These attacks have been going on for about the last month and it has built up to this situation," he said.
About 1.5 million people have been forced from their homes across the north of the east African country by the 18-year-old conflict between the shadowy LRA and government forces.
In February LRA rebels killed more than 200 civilians in an attack on a camp for internally displaced people near Lira town.
But aid workers said the recent raids in Adjumani appeared mostly to be attempts by the LRA to steal food and medicines.
Peter was attacked by LRA rebels
Ugandan army spokesmen could not immediately be contacted for comment, but have in the past suggested the LRA have been short of supplies since 2002 when Sudan allowed Ugandan forces to enter southern Sudan to attack LRA bases.
Targeting aid camps
Kallon said WFP staff, along with investigators from the UNHCR refugee agency and the Ugandan government, were in Adjumani and neighbouring Moyo district to assess the situation.
UNHCR said in a separate statement nine of the 12 rebel attacks reported this month had targeted refugee settlements.
More than 200,000 Sudanese refugees are living in Uganda after fleeing two decades of civil war in the country's giant northern neighbour.
The LRA, led by self-proclaimed mystic Joseph Kony, is known for its brutality, and aid workers say it has abducted tens of thousands of children as fighters and sex slaves.