The targeting of humanitarian workers, harassment of civilians and "unprecedented criminality" in the town of Nyala are part of the "dangerous pattern" of violence caused by the prolonged conflict, he said.
"While the daily rate of casualties from fighting has declined in recent months, the damage to the social and economic fabric in Darfur and the longer term costs of this conflict are steadily becoming clearer," Annan said in the report to the UN Security Council obtained on Tuesday by The Associated Press.
Fighting began when Darfur tribes took up arms in February 2003, complaining of discrimination and oppression by Khartoum.
The government is accused of unleashing the Janjawid militia against civilians in a campaign of murder, rape and arson.
Hunger and disease
At least 180,000 people have died - many from hunger and disease.
As of 1 July, Annan said, 3.2 million people were in need of humanitarian assistance and 1.9 million were living in crowded camps in Sudan, cut off from their traditional lifestyles "and vulnerable to psychological as well as physical insecurities".
Up to 1.9 million people are
displaced and living in camps
"I urge both parties to recognise that despite some stabilisation of the security situation in Darfur, at a deeper level, living conditions are steadily deteriorating," he said.
"The longer the parties allow this climate of no war, no peace to continue, the higher the price to be paid for restoring safety, dignity and prosperity to the lives of all people in Darfur."
The secretary-general demanded "urgent corrective action" by the rebels and the government.
"First, they must control their military commanders and reverse the slide into predatory warlordism and criminal behaviour," he said.
He noted that the government still showed no intention of disarming the Janjawid or other militias" and is yet to hold a significant number of them accountable for the atrocities of earlier months."
Rebels in Darfur took up arms in
Second, Annan said, the government and rebels had to prepare to reach an agreement at the next round of peace talks in Abuja, Nigeria, on 24 August on the divisive issues of sharing power and wealth "which are at the root of the conflict in Darfur".
Annan deplored the government's use of aircraft for observation during a retaliatory attack against Sudan Liberation Army rebels, saying this violated UN Security Council demands and its own pledge.
The government said it was responding to a 23 July rebel attack against two convoys escorted by security forces.
"The longer the parties allow this climate of no war, no peace to continue, the higher the price to be paid for restoring safety, dignity and prosperity to the lives of all people in Darfur"
The secretary-general blamed the Sudan Liberation Army and the Justice and Equality Movement - the rebel groups that first took up arms in Darfur, and last month signed a declaration of political principles with the government outlining a long-term solution to the Darfur crisis, for the "considerable rise" in July in abductions, harassment, extortion and looting.
The abduction of Sudanese working in non-governmental organisations reached "alarming proportions" at the beginning of July with 10 being held by the SLA, he said.
Ministry of Health teams conducting polio vaccinations were also abducted, he said.
Annan also complained that banditry continued to plague Darfur's highways, and the government had not done enough to prevent sexual violence and physical abuse of women and girls in the camps by armed men.
Meanwhile, a Sudanese panel set up to investigate the death of first vice president and former rebel leader John Garang in a helicopter crash is to visit the site on Thursday, a member of the panel said.
"The national committee will travel to crash site on Thursday and then proceed to (the Ugandan capital) Kampala," Siraj Eddin Hamid told AFP.
First vice president John Garang
died in a plane crash on 30 July
Garang died on 30 July when a Ugandan presidential helicopter on which he was travelling crashed on its way back to south Sudan.
Sudanese President Omar al-Beshir issued a decree on 8 August establishing the seven-member committee to investigate the crash.
The committee includes six aviation experts - three from the Sudanese government and three from the former rebel group Garang headed, the Sudan People's Liberation Movement (SPLM).