The comments came as the US defence department released previously classified statistics on the size of the US nuclear arsenal.
The department said that the US maintains 5,113 nuclear warheads in its stockpile and "several thousand" more retired nukes that await dismantling.
Officials say this is part of a US drive to prove it is serious about disarmament and transparency about its nuclear weapons.
Speaking at the same conference hours after Ahmadinejad's speech, Hillary Clinton, the US secretary of state, said Iran was diverting "attention away from its record in an attempt to evade accountability".
"Iran has defied the UN Security Council ... and placed the future of the non-proliferation regime in jeopardy," Clinton said, adding that "potential violators must know that they will pay a high price if they break the rules".
Clinton told reporters after her speech that Ahmadinejad would fail in his bid to derail the review conference.
"It appears that Iran's president came here today with no intention of improving the NPT," she said.
"He came to distract attention from his own government's failure to live up to its international obligations, to evade accountability for defying the international community ... but he will not succeed."
Ahmadinejad is the only head of state taking part in the conference, which is hosting delegations from 189 countries.
He urged "the suspension of members in the board of governors of the International Atomic Energy Agency, which use or threaten to use nuclear weapons".
"How can the United States be a member of the board of governors when it used nuclear weapons against Japan," Ahmadinejad said.
Alireza Ronaghi, Al Jazeera's correspondent in Tehran, said: "We have to be realistic about what he wanted to achieve.
"The walkouts were expected, they were announced before. He doesn't have any worries about the people who walked out, his message will come across and be heard despite the walkouts.
"He is under enormous pressure from people in Iran, who accuse him of being too soft, to sign out of the Non-Proliferation Treaty.
"It is a difficult balancing act that he is trying to carry out."
Speaking from New York, Marwan Bishara, Al Jazeera's senior political analyst, said: "The US' and Iran's obsessionwith each other is paralysing the international community, is hindering the process from moving forward.
"There's a lot of dragging and there is a long way to go. Those who have the nuclear weapons are insensitive to those who do not have them.
"His [Ahmadinejad] message was that 'we are the victims, they [the nuclear powers] are the aggressors'.
"The forum is at risk of turning into a circus where the US and Iran attack each other while leaving the rest of the international community outside the cage."
Tehran has refused to abandon its enrichment programme and now faces the prospect of UN-backed sanctions as a result of its defiance in the face of international pressure.
The month-long NPT review conference is held twice a decade to discuss the implementation of the treaty, which calls on signatories to abandon nuclear weapons and to prevent the spread of the technology.
Ban Ki-moon, the UN secretary-general, said over the weekend that more progress needs to be made in disarmament efforts, which have been boosted by new pledges from the US and Russia in recent months.
Ban also said Iran's and North Korea's nuclear programmes were "of serious concern to global efforts to curb nuclear proliferation".