B'Tselem, a body that monitors human rights in the West Bank and Gaza, said in a report released on Thursday that most of the 9,000 Palestinians being held by Israel were illegally imprisoned inside the Jewish state.
"The vast majority of Palestinian prisoners are held in prisons inside Israel, and not in the occupied West Bank,in contravention of international humanitarian Law," said B'Tselem's communications director, Sarit Michaeli.
"In order to guarantee basic human rights, they should be transferred to prisons in the West Bank where Palestinians are allowed to travel," she said.
Israel occupied the West Bank and Gaza in the 1967 Middle East war and although it withdrew soldiers and Jewish settlers from Gaza in August last year.
Since it is often very difficult for Palestinians to get permits to enter Israel, prison visits end up taking place only every few months.
Checks and delays when they do happen can mean that a visit of a few hours can take up to 24 hours to carry out.
"Israel's arbitrary and disproportionate policy not only infringes the right to family visits, it also results in violation of other rights and principles of international humanitarian and human rights law," said the report, entitled "Barred from Contact".
Israel's prison system said in response it made all efforts to allow family visits to take place in a "respectful manner".
"(The prison system) also acts in accordance with laws and regulations to check proof of identity meticulously, especially in light of recent events," said a spokesman, Yaron Zamir, referring to the stabbing of a guard by a prisoner's relative.
In a response sent to B'Tselem in August, the Justice Ministry said it did everything possible, within the constraints of security, to facilitate visits and said that it had denied only 41 of 4,616 visits requested since December 2005.
"The state has been acting relentlessly, despite the many security and administrative difficulties involved, to enable the existence of these visits," it said.
The ministry did not directly address the allegation of the illegality of holding the prisoners in Israel.