Jan Pronk said on Wednesday that a vicious circle of violence in Darfur could cause the collapse of a ceasefire signed in April between two rebel groups and the government.
"What is lacking is a sense of urgency... the commitment to do it [reach an agreement] as soon as possible," he said.
Government and rebel delegates filed into African Union-sponsored talks in Nigeria on Wednesday, but for a third consecutive day rebels called for an adjournment within an hour of the start of talks.
Observers said the two rebel groups, who have flown in from isolated locations across Africa, were still formulating their position at the talks, aimed at ending the conflict that has displaced more than 1.5 million people.
Khartoum has expressed impatience at the stalling tactics.
"We feel they are wasting our time and I think we have been patient enough. I think this should be their last chance to show whether they are ready to negotiate," said government spokesman Ibrahim Muhammad.
The United Nations says Darfur is one of the world's worst humanitarian crises, which has killed around 70,000 people through disease and malnutrition since March.
There are no reliable figures for how many people have died due to fighting.
"We feel they [the Darfur rebels] are wasting our time and I think we have been patient enough. I think this should be their last chance to show whether they are ready to negotiate"
Sudan government spokesman
Pronk added a UN Security Council resolution demanding that the violence stop in Darfur and that a humanitarian protocol be drawn up in April meant the two sides need not discuss the issues of aid access and security.
These two issues had stalled the previous round of talks in Abuja and continued to block progress in the Nigerian capital this week.
"Don't discuss it any more - just do it so that you can discuss political issues, political objectives," he said.
The Sudan Liberation Army, the main rebel group, insisted on Wednesday that the government must agree to disarm militia before they would be ready to discuss political issues.
After years of skirmishes between nomads and farmers over scarce resources in arid Darfur, rebels took up arms accusing Khartoum of neglect and of using mounted Arab militias, known as Janjawid, to loot and burn non-Arab villages.
Khartoum denies any link to the Janjawid, calling them outlaws.
The Darfur rebels say Khartoum
is guilty of genocide
It also says the conflict has been wrongly labelled "a racial war" when it is essentially a dispute over land.
The United Nations has threatened Sudan with possible oil industry sanctions if it fails to stop the violence.
Pronk, who leaves on Wednesday for New York to give a monthly briefing on Darfur, said ordinary civilians and aid workers were suffering as talks dragged on.
"Insecurity and violence and violation of human rights is on the rise. We are hardly able to stop it and parties do not seem to be willing to stop it on the ground," he said.
Pronk added the rebels, and not the government, were impeding aid access to the diseased, hungry and destitute refugees.
He also warned if the talks in Abuja did not make progress, the ceasefire, which each side has accused the other of breaking, could collapse in Darfur.
"Insecurity and violence and violation of human rights is on the rise. We are hardly able to stop it and parties do not seem to be willing to stop it on the ground"
UN envoy to Sudan
"It is officially still a ceasefire, but in practice there are constant attacks," he said.
Rebels said 26 civilians died in a bombing raid on Wednesday morning in Allaiet, in eastern Darfur,
But the government said the attack happened three days ago, its forces were only responding to a rebel attack and that those killed were rebel fighters.