But in Tokyo, defence officials said the base was not hit and they were still checking details of the attack that occurred on Tuesday.
"One thing that we are sure of is that the mortars have not hit the Japanese base," a Defence Ministry spokesman said. The ministry believes the bombs landed "several hundred metres" from the base.
It was not immediately clear who launched the attack, but fighters loyal to firebrand Shia cleric Muqtada al-Sadr have revolted against US-led occupation troops and the country's interim government in many cities in southern Iraq and Baghdad suburbs.
The sources said Iraq police had begun rounding up suspected fighters of al-Sadr's al-Mahdi army in Samawa.
Attacks on the Japanese contingent have been rare.
Last month Japan said it would keep its troops in Iraq despite threats of attacks - purportedly from a group led by Abu Musab al-Zarqawi - if Japan did not withdraw its forces.
The purported statement from al-Zarqawi, an Islamic dissident with suspected ties to al-Qaida, was posted on an Islamist website and demanded Japan should follow the Philippines and pull its troops out. A later internet message disowned the warning.