Addressing a meeting on the United Nations and the news media on Thursday, Annan pointed to "the fact that the bulk of the money that Saddam Hussein made came out of smuggling outside the oil-for-food programme, and it was on the American and British watch".
"Possibly they were the ones who knew exactly what was going on, and that the countries themselves decided to close their eyes to smuggling to Turkey and Jordan because they were allies."
Annan's strong remarks come only hours after US prosecutors charged three people with scheming to pay millions of dollars in kickbacks to Saddam Hussein's government out of oil-for-food funds, which were meant to purchase relief supplies for Iraqis.
On Thursday morning, American David Chalmers and Bulgarian national Ludmil Dionissiev were arrested in Texas, US attorney David Kelley said. Extradition of the third defendant, British oil trader John Irving, was sought from Britain.
"Possibly they (the US and UK) were the ones who knew exactly what was going on, and that the countries themselves decided to close their eyes to smuggling to Turkey and Jordan because they were allies"
UN Secretary-General Kofi Annan
If convicted, the three men could each face a maximum term of 62 years in prison.
The oil-for-food scandal is currently being investigated by a committee headed by former US Federal Reserve chairman Paul Volcker.
The committee has so far issued two interim reports which charged the programme's director Benon Sevan with unethical behaviour and raised serious questions over the dealings of Annan's son, Kojo.
"One had hoped that when Volcker comes out with his final report in June, putting things in perspective, this will die down.
"But I don't think with that group it will die down," Annan said.
The oil-for-food programme, the largest aid programme in UN history, was initiated in 1996 to alleviate the plight of the Iraqi people who had been weathered down by 13 years of sanctions and an economic embargo.
Since then, the programme has overseen $64 billion in deals as the United Nations supervised Iraqi oil sales and the purchase of humanitarian supplies with the revenues.