Speaking to 1,000 Muslim leaders on Thursday, Surayud pledged to investigate the disappearance of Muslims and to root out corrupt and abusive officials in the three southern provinces.
"I'm here today to apologise for what past and present governments have done," Surayud said in the southern town of Pattani.
"I come here today to reach out to everyone and say: I am sorry," Surayud said.
He admitted that, as a former army chief, he had failed to counter the oppressive actions of Thaksin Shinawatra, the former president.
In 2002, Thaksin dissolved a 21-year-old multi-agency body once credited with keeping stability there, saying the low-key separatist movements of the 1970s and 1980s had come to an end and the violence was caused mainly by "petty thugs".
After the Southern Border Provinces Administration Centre (SBPAC) was disbanded, unrest brewed and exploded in January 2004 when there were a number of arson attacks on schools across the region and raids on an army camp.
Thaksin sent thousands of troops and police to the region, a former sultanate annexed by Bangkok a century ago.
According to analysts, hundreds of suspects were abducted and tortured.
Surayud said those days were over, while appealing for help to end violence in which more than 1,700 people have been killed.
"The legal apparatus, from police to prosecutors, has to be revamped," he said. "Cases of missing people, have to be resolved quickly. We don't know how many people have gone missing in the entire country."
The justice ministry was considering dropping criminal charges against 58 Muslims involved in a 2004 protest which led to the death of 78 people in army custody, Surayud said.
Surayud's visit came a day after the government revived the SBPAC, which focused on rural development and investigated complaints of corruption and injustice, and appointed a Buddhist southerner to head the agency.
General Sonthi Boonyaratglin, who led the September 19 coup that toppled Thaksin, said on Wednesday the SBPAC faced a difficult job to restore peace in a region where daily shootings and bomb attacks had not stopped since the coup.