The prisoners are being held in legal limbo under a practice known in Israel as "administrative detention", which authorises the army to order the arrest and detention of Palestinians without allowing them access to the legal rights they could expect under civilian law.
Israel says the practice complies with international law and is a necessary measure based upon "security considerations".
But the organisations say that the prisoners find it "impossible" to challenge their detention because the evidence against them is often kept secret by the Israeli judges who review the cases.
"The judicial review of the administrative-detention proceedings presents a semblance of a fair judicial process, but in fact denies the detainees any possibility to reasonably defend themselves against the allegations made against them," the groups said in a statement.
Over an eleven month period, judges approved imprisonment in more than 95 per cent of the cases that they heard, often refusing to reveal the evidence that led the military to deem the detainee a threat to the "security of the region".
Under international law, a state may legally detain a resident of occupied territory without trial if it believes there is a specific threat posed by the individual's continued liberty.
But the rights groups say Israel is using the law to detain prisoners outside of civilian judicial processes in a bid to keep evidence secret, and that this is illegal.
"In the vast majority of cases, the individual is held in administrative detention for no longer than two years"
Israeli military statement
"An entire legal system that routinely imposes privilege on most of the evidence cannot be justified," the report said.
The Israeli military deny the charge.
"Administrative detention is applied as a last resort, in order to remove terror activists regarding whom there is concrete information concerning a clear and present danger posed by them to the security of the area.," it said in a written statement to Al Jazeera.
The statement also denied that the majority of prisoners were being held for many years.
"In the vast majority of cases, the individual is held in administrative detention for no longer than two years," the statement said.
The rights groups' 70-page report calls on Israel to release those held in administrative detention, or charge them "in accordance with the standards of due process specified in international law".