Nouri al-Maliki, the Iraqi prime minister, welcomed Thursday's transfer but said armed groups would try to carry out attacks or incite civil disturbances with the withdrawal of the foreign troops from the area.
"It is a great national day that will be registered in the history of Iraq. This step will bring happiness to all Iraqis," al-Maliki said during a handover ceremony in the provincial capital of Samawah.
"Be sure that the terrorists want to destroy and foil the process of taking over the security issue and to hamper the political process and the national unity government.
"They will not spare any efforts to destroy this step and ensure that no further steps are taken. But, with solidarity and patience, you will cut off the hands that want to sabotage this area," he added.
Iraqi forces holding their nation's flag led a parade of troops marching in formation past the prime minister and other dignitaries at a stadium in the city, about 370km southeast of Baghdad.
Japan is in the process of withdrawing its 600 non-combat troops that were based near Samawah, ending Tokyo's largest military deployment in the postwar era, and the first to a war zone since 1945.
"Be sure that the terrorists want to destroy and foil the process of taking over the security issue and to hamper the political process and the national unity government"
Nouri al-Maliki, the Iraqi prime minister
The international troops have said they would maintain a presence nearby and be prepared to help the Iraqis if needed.
Australia's defence ministry has said its 460 troops in the province will be transferred to an air base in Tallil, about 100km southeast of their current posting in Samawah.
Britain has about 170 soldiers in the province, a fraction of its 7,200 service members who are based in southern Iraq.