A resolution to re-examine the issue at future sessions, brought by the Organisation of the Islamic Conference (OIC), easily won passage at the 47-member forum on Friday.
A second OIC resolution, expressing deep concern at an "increasing trend of defamation of religions" and incitement to religious hatred, was also adopted along similar voting lines.
The two votes, on the final day of the Rights Council's inaugural two-week session, were seen as divisive.
Many states and rights activists had hoped that all decisions would be taken by consensus to avoid the acrimony that marked the council's predecessor, the UN Commission on Human Rights, which also debated Palestinian issues at each session.
Meanwhile, Israel kept up an offensive in Gaza on Friday, when warplanes set ablaze the interior ministry offices on the third day of attacks after an army corporal was seized in a cross-border Palestinian raid last Sunday.
The vote to examine the situation in the Palestinian territories at future sessions passed with 29 countries in favour, 12 against, five abstentions and one delegation absent.
The resolution also called for existing UN human rights investigators, known as special rapporteurs, to report on the situation in the territories at the next session in September.
Regional powers, including Egypt, Iran, Jordan, Lebanon, Saudi Arabia and Syria were among the resolution's sponsors.
Western countries, including Britain, Canada, France and Germany, as well as Japan, voted against the text.
The United States did not stand for election and only had observer status.
Itzkhak Levanon, Israel's ambassador, said in a speech before the vote that the OIC text was "imbalanced and intentionally one-sided".