Some of the vessels had mangled guard rails and large scratches in their sides as a result of heated run-ins with a boat from the Australian-based Sea Shepherd Conservation Society, the agency said.
It said bad weather and the skirmishes with anti-whaling activists had meant the fleet could not carry out whaling for a total of 16 days.
"This season's catch was reduced as a result of the interference by protesters," Shigeki Takaya, an agency spokesman, said.
The Sea Shepherd which chased the Japanese fleet more than 3,200km through the Antarctic ocean withdrew in February after increasingly hostile exchanges between the two sides.
|Japan says the hunt is to further research on the mammals [AFP/Australian Customs]
Captain Paul Watson, a spokesman for the anti-whaling campaign, said the Steve Irwin broke off the chase because the risk of serious injury at sea was too great.
But the operator of the Japanese whaling fleet accused the activists for escalating the attacks by ramming two Japanese vessels and pelting whalers with acid-filled glass bottles.
Kazuo Yamamura, the president of Kyodo Senpaku Kaisha, the company which runs the fleet, said he was upset by damage allegedly caused to his ships by Sea Shepherd.
"I'm enraged, and my blood is boiling with anger," he told reporters on Monday.
Japanese officials have also described the environmentalists as "terrorists" and "pirates", and demanded the Australian take action against Sea Shepherd.
Japan officially stopped commercial whaling after agreeing to a global moratorium in 1986, but began what it calls a scientific research whaling programme the following year.
While whale meat is no longer a common food in the country, meat from the hunt is sold in Japanese supermarkets and upscale restaurants.
Whaling for research purposes is allowed under a loophole in the regulations overseen by the global whaling watchdog, the International Whaling Commission.
Japan has also accused Western nations of not respecting what it says is its long history and culture of whaling.