"We heard one big boom at first and then we were chased by the group for about an hour," a crew member told Japanese public broadcaster NHK. "We were attacked on the left and the right sides of the ship at least four times."
The attack occurred in international waters some 440km east of the Yemeni port of Aden, Japanese officials said.
The tanker's crew included 16 Filipinos and seven Japanese, the shipping company said.
Noel Choong, head of the Malaysia-based International Maritime Bureau's Piracy Reporting Centre, told AFP that "pirates on five speed boats" attacked the tanker.
Choong said the pirates could have used rocket-propelled grenades and machine gun fire. "But the tanker increased speed and managed to prevent the pirates from boarding and taking control of the oil tanker."
The tanker was on its way to the Saudi port of Yanbu in the Red Sea after unloading oil at the South Korean port of Ulsan when the attack happened, the company said.
A Yemeni maritime source said the tanker has been diverted to Aden for repairs caused during the incident.