The Nihon Keizai Shimbun poll found on Thursday that 61% of Japanese people want troops home now.
The majority of those who took part voiced growing opposition to Japan's first deployment to an active combat zone since the second world war.
The business daily also revealed that nearly three-quarters of ruling Liberal Democratic Party supporters were extremely concerned about the growing instability in Iraq.
The same paper carried out a poll last April in which only 42% called on the government to withdraw forces from Iraq.
Japan's deployment is meant to end on 14 December unless Prime Minister Junichiro Koizumi extends it, as he has indicated he will - despite widespread instability and the abduction and beheading of a Japanese traveller last month.
Just 25% of those who took part favoured extending the deployment – with some of those voters not stressing the necessity to rebuild Iraq but to maintain Japan's close alliance with the US.
The poll also showed overall support for Koizumi's cabinet stood at about 44%, unchanged from the last such survey in September.