The Haaretz newspaper poll found that 56% of Israelis were against the proposal - even if an agreement with the Palestinians could not be reached.
The survey showed that 37% supported the plan which has been put forward by Ehud Olmert, the Israeli prime minister.
Despite opposition to the plan, 51% of Israelis thought the plan would be implemented in the next four years, the poll said.
Olmert's plan would see Israel withdraw from some smaller West Bank settlements while cementing its hold on large areas of land illegally occupied under international law.
The Kadima party leader has said he will try to reach an agreement with Mahmoud Abbas, the Palestinian president, although he has also pledged to act unilaterally unless the governing Hamas movement renounces violence and recognises Israel.
A similar poll in February showed that 60% of Israelis backed the move although Friday's survey did not say why support for it had fallen.
Responding to the poll, Shimon Peres, the Israeli deputy prime minister, called for Israel to renew stalled peace talks with Abbas, but said that Israel need not abandon the plan altogether.
"I think that now we should stress bilateral talks, that's the real situation. I hope we are serious people and want to exhaust what we can do by bilateral agreement," Peres said.
A representative sample of 515 people were questioned in the poll, which was supervised by Tel Aviv University.