Himself a Maori, Horomia, speaking on Radio New Zealand, said: "These articles imply different classes of citizenship where indigenous people have a right of veto that other groups or individuals do not have."
Mal Brough, Australia's indigenous affairs minister, said the declaration was "outside what we as Australians believe to be fair".
Under negotiation for 20 years, the document says that indigenous peoples, whose number has been put at 270 million worldwide as understood by the declaration, "have the right to self-determination".
Phil Fontaine, chief of the Canadian National Assembly of First Nations, called the declaration a "huge victory for our indigenous peoples worldwide".
Speaking to Al Jazeera from New York, he said: "One of the significant considerations ... is that the international community now recognises the validity and legitimacy of our right to land and resources, our cultures, our languages, our human rights."
One of its most controversial articles states that "indigenous peoples have the right to the lands, territories and resources which they have traditionally owned, occupied or otherwise used or acquired".
That could potentially put in question most of the land ownership in countries, such as those that opposed the declaration, whose present population is largely descended from settlers who took over territory from previous inhabitants.
A balancing clause inserted at a late stage in the text says nothing in it can authorise or encourage "any action which would dismember or impair, totally or in part, the territorial integrity or political unity" of states.
The additional clause was still not adequate for the four objectors, notably Canada.
Many of Canada's one million aboriginal and Inuit people live in overcrowded, unsanitary housing and suffer high rates of unemployment, substance abuse and suicide.
"The provisions in the declaration on lands, territories and resources are overly broad, unclear, and capable of a wide variety of interpretations," John McNee, Canada's UN ambassador, said.
Robert Hagen, the US delegate, said the UN Human Rights Council, which prepared the text, had not sought consensus.
"This declaration was adopted ... in a splintered vote. This process was unfortunate and extraordinary," he said.
Aside from land-ownership issues, critics also criticised provisions on indigenous peoples' intellectual property, right to be consulted on laws affecting them and right to exempt their land from military activities.
But supporters of the declaration said it gave long overdue recognition to indigenous peoples.
Fontaine said: "What it does, is it sets out standards. And we will be able to assess and measure progress in so many different areas, whether we're talking about the fair and just resolution of land claims, whether we're talking about access to resources and the preservation of our languages and our cultures."
Most US allies, including Britain and Japan, also voted for the declaration, saying last-minute amendments had made it acceptable, given that it did not have the force of international law.