"No one understands ... how over-consumption by obese people in the world costs $20bn each year," Jacques Diouf said.
He said that indirect costs resulting from premature deaths and associated diseases total $100bn.
Diouf said that $1.2trn was spent on weapons in 2006 while aid to agriculture stood at $3.4bn in 2004.
"In real terms, the share of agriculture in public aid to development has fallen, from 17 per cent in 1980 to three per cent in 2006," he said.
He said that a goal set by the 1996 world food summit in Rome for reducing the number of hungry people in the world by half by 2015 was a long way off being reached.
"With current trends, the summit's goal will be attained in 2150 instead of 2015," he said.
Mark Seddon, Al Jazeera's Europe correspondent said, "Ban Ki Moon wanted to focus attention on what he called historic underinvestment in agriculture."
"Between $15-$20 billion are needed to begin to try and get the same results that the world got with the green revolution in India. [Ban] wants to see that taking place in Africa and parts of Asia."
Robert Zoellick, the head of the World Bank, is saying there needs to be an immediate lifting of export bans, Seddon said.
"There are a lot of issues such as biofuels, speculation on the markets; these sorts of issues haven’t really been addressed here."
Rising food prices across the world are being blamed on myriad factors, including high oil prices, population growth and the diversion of crops from food production to fuel production.
The World Bank says that in the first quarter of 2008, prices of all major food commodities reached their highest levels in nearly 50 years.
UN says soaring price of basic foods such as rice and cereals could affect about 100 million of the world's poorest
Global rice stocks have halved since hitting a record high in 2001 while demand is continuing to rise
In Asia, rice prices have almost tripled this year alone
Financial speculators, rising populations, floods, droughts, increased demand from developing countries, and removing crops from the food chain to produce biofuels have been cited as factors
Price rises have led producing nations to enforce export restrictions, further putting the squeeze on supply, especially in countries relying on imports
Ban has called on the US and other nations to phase out subsidies for food-based biofuels.
The subsidies have encouraged many farmers to grow crops for energy use rather than for human consumption.
The presence of Robert Mugabe, Zimbabwe's president, at the summit on Tuesday sparked condemnation from Australia and Britain.
Mugabe, whose Zanu-PF party has seized land from white farmers in Zimbabwe, said he was undoing a legacy left by former colonial "masters", meaning Britain.
An appearance at the summit by Mahmoud Ahmadinejad, Iran's president, was also condemned by some attendees.
Ahmadinejad accused unnamed "powers" of profiteering from high food prices through subsidies and market manipulations.