The five drowned late on Monday in the floods that swept through Dhamar, 70km south of Sanaa, the official said, requesting anonymity.
Rescue efforts were continuing on Tuesday, and a main road linking Sanaa with cities to the south remained closed, he said.
About 1900 people were still trapped in around 100 flood-besieged homes, said Abdul Salam al-Ahsab, Dhamar's Red Crescent chief.
Rescue teams were trying to reach the trapped residents, he added.
The flooding occurred at the start of Yemen's rainy season. In April, 10 people were killed in torrential floods.