Museveni said settlements in the flood valley of the nearby River Manafa had left many people particularly vulnerable, and also blamed farmers for stripping the land clear of thick plant life that retains water.
At least four people were rescued from the wreckage on Wednesday, but more than 250 people are still missing, an official from the Uganda Red Cross told the AP news agency.
"We expect to recover more bodies as time goes on. But the exercise is slow because we are using hoes to dig the dead bodies out of the thick mud," Kevin Nabutuwa said.
Malcolm Webb, a journalist in Uganda, told Al Jazeera hundreds of people were helping to dig bodies out of the mud.
"There's been about 200 Ugandan army soldiers and tens of staff from the Uganda Red Cross [helping to dig].
"But still most of the digging been's done by volunteers from the community, and mostly done with tools, with sticks, hoes and spades."
He said getting heavy machinery into the area was not possible, with the villages being two hours walk along a muddy track from the nearest road.
The landslide, which hit on Monday night after seven hours of rain, engulfed the village of Nametsi near the Kenyan border, burying houses, people and livestock.
The government said three villages with more than 3,000 residents were badly hit and mudslides were feared in five other districts experiencing deluges.
Parts of Uganda and neighbouring Kenya have had sustained rainfall over much of the past two months, which is usually a dry period between rainy seasons, and floods are already plaguing large areas.
Unusually heavy rains also battered eastern Uganda in 2007 and forced 2,000 people from their homes and affected 50,000 people in what humanitarian officials said were the worst rains in 35 years.