In a report on two days of closed-door discussions with an Israeli delegation last month, the UN Human Rights Committee (UNHCR) also called for an end to "targeted killings" of Palestinian resistance fighters and to the destruction of homes where they lived.
An 18-member committee of independent experts said the wall "imposes additional and unjustifiably severe restrictions on the right to freedom of movement of...Palestinians within the Occupied Territories".
"The construction of...(the wall) within the Occupied Territories should be stopped," it said.
Israel says the barrier is needed to keep out bombers who have killed its citizens in retaliation for Israeli attacks during the Palestinian uprising for independence.
However, Palestinians call the project a land grab that prejudges final borders to be decided under a US-backed peace plan.
The committee's report, acknowledging what it called the seriousness of Israel's security concerns, said the fence had a negative impact on all aspects of life in the territories, especially access to health care and water.
As an occupying power (Israel) must also observe the terms of the (Covenant of Rights) in the Palestinian territories.
Because of this, the UNHCR said it was a violation of provisions of the 1966 International Covenant on Civil and Political Rights, which Israel has signed.
The body, which has no enforcement mechanism, sits every year to study how signatory states are observing the covenant. It studies reports by governments and from individuals claiming their rights under the pact have been abused.
Israel argues it is not bound by the covenant outside its own borders. But this is rejected by the committee, which says that as an occupying power it must also observe the terms of the pact in the Palestinian territories.
Israel at odds with UN
In a statement on the report, Israel's ambassador in Geneva Yaakov Levy said his government remained at odds with the committee on that issue.
But his statement welcomed what it called the body's understanding of "the necessary measures taken by Israel to defend its civilian population's right to life in the face of massive terrorist attacks and suicide bombings..."
In its report, the group - which has members from a wide range of countries including the United States, Britain, Australia and Canada as well as from African, Asian and Latin American countries - welcomed some rights advances in Israel.
These included improvements in the status of women, measures against trafficking in women, efforts to increase education levels among Israeli Arab, Bedouin and Druze communities, and the launch of a development plan for Arab areas.