Ephraim Zuroff, from the Simon Wiesenthal Centre, said: "Israel can't throw open the gates and allow unlimited access for people who are basically economic refugees."
But Eytan Schwartz, an advocate for Darfur refugees in Israel, said: "The state of Israel has to show compassion for refugees after the Jewish people was subject to persecution throughout its history."
He said about 400 refugees from Darfur had entered Israel in recent years.
"Oppression and injustice made those people look for salvation even if it was in Israel"
head of Egypt's Arabic Network for Human Rights Information
Human rights groups say more than 2,000 illegal migrants from Africa have entered Israel from Egypt over the past year seeking a better life.
In recent months, Israel has been grappling with how to deal with an increasing number of refugees. As many as 50 people arrived each day in June, according to the UN high commissioner for refugees.
"The policy of returning back anyone who enters Israel illegally will pertain to everyone, including those from Darfur," Baker said.
Israel handed over 48 Sudanese to authorities in Egypt on Saturday, Egyptian security officials said.
Baker put the number at 50 and noted the Israeli government had agreed to allow 500 refugees from Darfur already in Israel "to remain for humanitarian reasons".
Israel considers Sudanese refugees to be enemy nationals, partly because of Khartoum's hostility toward Israel, and several dozen are in Israeli jails.
Egypt has stepped up efforts to prevent illegal migrants crossing its border into Israel, a source of tension between the two countries.
Egyptian police shot at a group of African migrants trying to sneak into Israel in July, killing one woman and wounding four, including a child.
"We can blame them [the Sudanese refugees] but we should condemn the governments. Oppression and injustice made those people look for salvation even if it was in Israel," Gamal Eid, head of Egypt's Arabic Network for Human Rights Information, said.
Experts estimate that 2.5 million people have been driven from their homes in Darfur, and about 200,000 people have died, since 2003 when non-Arab fighters took up arms, accusing the Sudanese government in Khartoum of marginalising the region.