The 600-odd soldiers, who are on a humanitarian mission in Samawa, southern Iraq, would start withdrawing in March, said the Sankei Shimbun newspaper, which closely covers national security issues, without identifying its sources.
Junichiro Koizumi, the Japanese prime minister, would make the announcement after co-ordinating efforts with Britain and Australia, who are providing security for the Japanese troops.
The troops, who are banned from using their weapons under Japan's pacifist constitution, have suffered no casualties during their humanitarian assistance mission.
In December, Japan extended its Iraq mission by another year with an eye to begin pulling out as early as mid-2006.
Most Japanese oppose the deployment to Samawa, despite it being considered a relatively safe area within Iraq.