The storm tore through the remote northern town of Saroma on Tuesday, knocking out electricity to hundreds of homes, police said.
Local television showed a scene of devastation on the island of Hokkaido with a wide swath of collapsed buildings, badly damaged cars and utility poles strewn across streets.
A total of 52 people - 45 construction workers and seven residents from damaged homes nearby - were taking shelter at a town gymnasium on Tuesday night, said Hirofumi Matsumura, Hokkaido prefectural government official.
Twenty-five people were treated for injuries in hospital.
Local television networks estimated that the tornado was one of the strongest to hit Japan since the second world war, with wind speeds of 70m per second.
National broadcaster NHK quoted a local woman, Keiko Takeda, as saying that the skies suddenly darkened over the town and when she opened her window winds were swirling outside.
"It was very strong, but it was over very quickly," she said.
Other witnesses said there may have been two tornados and that after hitting the town they veered off into the surrounding hills.
According to the Central Meteorological Agency, the worst tornado previously recorded in Japan was just two months ago, when three people were killed on the southern island of Kyushu.