At least 110 landslides have been reported in Hofu, which lies about 1,000km southwest of Tokyo.
Rivers and canals were also flooded at more than 100 locations.
"It's all flooded now. I was very,very scared," said one local resident who managed to leave her house before it was swept away by massive floods.
Elsewhere in Hofu, an 85-year-old woman was buried in a separate landslide and found dead on Tuesday, and another person drowned after being washed into a swollen river.
The government has issued warnings for more landslides, rising river waters and flooding in the Yamaguchi area and sent in troops to help with the search.
"The Japanese government has been collecting local information since last night and we have deployed our Self Defense Forces to Yamaguchi," Takeo Kawamura, Japan's top government spokesperson, told a news conference on Wednesday.
According to Japan's meteorological agency, the peak of the rain has passed, but more than 25cm of rainfall was reported in 24 hours through to Tuesday evening.