Canada and Israel have said for months that they will shun the meeting, which the UN organised to help heal the wounds left by its last race summit in South Africa in 2001.
Call for 'fresh start'
The US and Israel walked out of that conference after Arab states sought to define Zionism as racist.
Barack Obama, the US president, said Washington wanted a "clean slate" before tackling race and discrimination issues at the UN.
"If we have a clean start, a fresh start, we are happy to go," he said at a news conference after the Summit of the Americas in Trinidad and Tobago on Sunday.
"If you're incorporating a previous conference that we weren't involved with, that raised a whole set of objectionable provisions, then we couldn't participate."
Australia said it shared US concerns about the declaration for the follow-up conference, which omits explicit references to Israel and the Middle East but "reaffirms" the text adopted at the 2001 Durban summit which singled Israel out.
Fears of 'misuse'
|Rights groups have urged nations to attend the talks and "engage against racism" [Reuters]
Frank-Walter Steinmeier, Germany's foreign minister, said Berlin believed "the conference will be misused for other interests, just as the previous conference was in 2001".
Britain and the Czech Republic, which currently holds the rotating EU presidency, will attend the meeting, but send only low-level delegations headed by their ambassadors in Geneva.
Activist groups including the International Federation of Human Rights and Human Rights Watch urged European leaders not to follow the US lead but contribute to the conference on topics such as how to prevent xenophobic attacks.
"We regret the absence of some states ... and urge other governments which have not yet confirmed their decision to constructively participate in the review conference and demonstrate the willingness to engage against racism," they said in a joint statement.
Navi Pillay, the UN high commissioner for human rights, has called on governments to use the meeting to find ways to ease ethnic and racial tensions that threaten migrant workers and minorities, and could worsen amid the global economic downturn.