Annan, opening an international aid conference for Iraq, urged Iraqi leaders to overcome sectarian and regional tensions by seeking consensus on unresolved constitutional issues such as federalism and revenue-sharing.
"If current patterns of alienation and violence persist much longer, there is a grave danger that the Iraqi state will break down, possibly in the midst of full- scale civil war," he said on Monday.
Asked about Annan's comments, Barham Salih, Iraqi's deputy prime minister, said that his government faced many security challenges and obstacles "but that does not mean we are facing a civil war."
Annan's warning came as US commanders braced themselves a surge of attacks by Iraqi insurgents next week to mark the start of Ramadan, the most important month in the Islamic calendar.
US and Iraqi officials have in the past said that the increasing sectarian violence in and around Baghdad was pushing Iraq ever closer to all-out civil war.
Iraq's neighbours, Saudi Arabia, Iran and Turkey also voiced their concerns on Monday that Iraqi sectarian and ethnic tensions could spill over into the region.
"What we fear today is that the wise could fall in the traps of the ignorant, in which case Iraq, its unity and people would be victims," Prince Nayef, Saudi's interior minister, said in Jeddah on Monday, at a nine-nation meeting to discuss efforts to quell sectarian violence.
"The dangers of such a situation are not jeopardy to Iraq alone, but they will have an impact on the security of the international community and [Iraq's] neighbours," he said.
In an apparent reference to violence between Sunni and Shia Muslim, Prince Nayef also said that Iraqis should reject calls to divide Iraq along sectarian or ethnic lines.
The ministers from Iraq's neighbouring countries agreed to form a liaison unit in Baghdad and pledged to share intelligence, crack down on smuggling and help in training officers.
Saudis fear jihadists' return
Iraqi officials at the meeting complained that Iran and Syria are not doing enough to block the flow of Islamist militants, who cross into Iraq to fight US led forces.
Alaa al-Ta'i, an Iraqi minister, said: "We need to improve the security situation in Iraq with more border control. We want logistical support including more equipment and training of Iraqi police in [other] countries."
Officials in Saudi Arabia, which has a 1,000 km desert border with Iraq, has already talked of their concerns that Saudis who have gone to fight alongside Iraq's Sunni insurgents could return to fight in Saudi Arabia.
Estimates of the number of Saudi militants in Iraq range from several hundred to several thousand, officials say.