The mudslide occurred on Sunday morning when many people were still asleep and the village was more populated than usual because many children had returned to celebrate Father's Day in Taiwan on Saturday.
Rescuers arrived on Wednesday after battling through washed-out roads, unstable terrain and bad weather for days, but found little in the village left to rescue.
They left soon after to search for survivors from other villages in the surrounding area.
At least four villages in southern Taiwan were devastated by mudslides and flooding after Morakot dumped more than two metres of rain on the island.
For more than 700 survivors on Wednesday, there was some relief as helicopters began airlifting them out of the disaster zone after they were found alive overnight.
"We have found around 700 people alive in three villages last night and 26 more this morning. We are deploying 25 helicopters to evacuate them," Major-General Richard Hu, a spokesman for the military, said.
It had been feared the villagers may have been buried by mudslides but the survivors made it to higher ground before walls of mud and rock covered their homes, officials said.
But many people remain unaccounted for in the remote region, where normally difficult access has been made even tougher by the floods and mudslides.
The government has confirmed 70 deaths from the disaster so far and the military has flown more than 100 helicopter trips to ferry out survivors, but the process has been painfully slow and dangerous.
|Mudslides are feared to have buried hundreds of people in remote mountain villages [EPA]
Highlighting that danger on Tuesday, three rescue workers were feared dead after their helicopter, which had been delivering food and rescuing villagers, crashed into a mountain near Wutai in Pingtung county in heavy fog.
Villagers in the south appeared to have been taken by surprise by the Morakot's arrival as the typhoon had been forecast to head north, but instead turned south to unprepared areas, our correspondent said.
Morakot, which had earlier killed at least 21 people in the Philippines, triggered the worst flooding in Taiwan in 50 years over the weekend before moving on to China.
It later weakened to a tropical storm but still wreaked havoc on China's southeastern coast, triggering a massive landslide in Zhejiang province that toppled a four-storey apartment building and killed two women late on Monday.
The storm also sparked the evacuation of about 1.4 million people from Zhejiang and neighbouring coastal province Fujian.