A mortar shell was found at the suspected site of Thursday's blast near the provincial council building in Samawa, Japan's Kyodo news agency reported, citing police.
There were no reports of injuries following the early morning explosion, it said.
Police believe the explosion may have been related to a clash earlier this week between police and protesters and to a rocket attack near the council building, in which one person died and six others were wounded, Kyodo said.
Japanese Prime Minister Junichiro Koizumi said on Thursday that he was worried about recent violence around Samawa.
"I am concerned. I want our troops to ensure their own safety," he said.
Japan has more than 500 troops near Samawa on a non-combat humanitarian mission that ends in December.
An explosion last week that damaged a Japanese military vehicle in Samawa led troops to temporarily suspend their activities, which include rebuilding roads and schools and supplying clean water to residents.
That incident intensified criticism of the mission, and prompted Japanese opposition lawmakers to step up pressure for an early withdrawal.
Polls indicate the Japanese public is deeply divided over the country's troop dispatch to Iraq.
Gas pipeline hit
Meanwhile, Aljazeera has learned that the natural gas pipeline near al-Yusufiya was blown up by unknown assailants.
Residents said a group of men were seen planting an explosive device under the pipeline that takes gas to a storage unit that fills cylinders for home usage.
US helicopters were seen flying over the scene and locals were worried the blast may have created a severe environmental hazard.