In his strongest defence to date on the deepening scandal over the UN-run oil-for-food programme in Iraq, Annan on Wednesday said UN officials were being blamed for a variety of misdeeds that they had no way of controlling.
The secretary-general also rejected conflict-of-interest charges involving his son.
"We had no mandate to stop oil smuggling,"Annan said. "They were driving the trucks through northern Iraq to Turkey. The US and British had planes in the air. We were not there."
He called some of the comments he read "constructive and thoughtful".
But he said: "Others have been outrageous and exaggerated. In fact, when you look at it, if you read their reports, it looks as if the Saddam regime had nothing to do with it. They did nothing wrong. It was all the UN.
"Why is all this being dumped on the UN?" Annan asked, recalling the supervisory role of the Security Council.
The General Accounting Office, the investigative arm of US Congress, says Iraq was estimated to have smuggled $5.7 billion in oil outside the UN programme. It said Iraqi elites pocketed around $4.4 billion by illegal surcharges on oil.
The UN-run oil-for-food programme, which began in late 1996 and closed last year, allowed Iraq to sell oil and buy civilian goods to ease the impact of the 1991 Gulf war UN sanctions on ordinary Iraqis.
Annan reiterated that if corruption against any UN official proved true, he would not hesitate to lift diplomatic immunity.
He has appointed a three-member panel to probe the allegations.
Among the media charges was a conflict of interest because Annan's son, Kojo, worked for the Geneva-based firm Cotecna.
The company, in December 1998, was awarded a contract to monitor Iraqi imports under the oil-for-food programme.
Annan said his son had joined Cotecna "before I became secretary-general, as a 22-year-old trainee".
"Neither he nor I had anything to do with the contract with Cotecna," Annan said.