"This is not a break in relations, but an expression of Israel's discontent for the lax Swiss attitude towards Iran," an Israeli foreign ministry official told the AFP news agency on condition of anonymity.
France has warned that European delegates would walk out of the meeting if Ahmadinejad makes "racist or anti-Semitic accusations".
"We will have to be very clear. We will not tolerate any slips," Bernard Kouchner, the French foreign minister, told France Info radio.
The Paris-based European Jewish Congress said in a statement that Ahmadinejad's presence meant the United Nations had "put the fox in charge of the hen house".
The speech by Ahmadinejad, who is a frequent critic of Israel and has cast doubt on the extent of the killing of Jews during the second world war, coincides with Holocaust memorial events being held in Israel and Europe.
Germany, Australia, the Netherlands and New Zealand joined the US on Sunday in deciding not to attend the meeting.
Washington cited "objectionable" language in a text prepared for the Geneva meeting as its reason for staying at home.
Opening the summit on Monday, Ban Ki-moon, the UN secretary-general, defended the disputed draft text as "carefully balanced".
"I deeply regret that some have chosen to stand aside. I hope they will not do so for long," he said.
"Some nations who by rights should be helping us to forge a path to a better future are not here"
"Some nations who by rights should be helping us to forge a path to a better future are not here. Outside these halls, interest groups of many political and ideological stripes shout against one another in acrimony."
Ban said that the meeting was important to tackle the threat of rising intolerance during the global economic downturn.
The draft does not mention Israel by name, but it reaffirms the Durban statement and its reference to the plight of Palestinians.
Canada and Israel have said for months that they will shun the meeting, which the UN organised to help heal the wounds left by its last racism summit in South Africa in 2001.
The US and Israel walked out of that conference after Arab states sought to define Zionism as racist.
Barack Obama, the US president, said Washington wanted a "clean slate" before tackling race and discrimination issues at the UN.
"If you're incorporating a previous conference that we weren't involved with, that raised a whole set of objectionable provisions, then we couldn't participate," he said at a news conference after the Summit of the Americas in Trinidad and Tobago on Sunday.
Navi Pillay, the UN human rights chief who is hosting the conference, said she was "shocked and deeply disappointed" by Washington's decision.
She said that some countries were focusing solely on one or two issues to the detriment of the fight against intolerance.
Several Muslim nations at the Geneva summit have called for moves to prevent perceived insults to Islam, which they say have proliferated since the attacks in the United States on September 11, 2001.