Rescue attempts continued through the night to save children buried in the rubble.
No survivors have been found since four people were pulled from debris on Saturday.
Disaster experts, officials and rescue workers had refrained from using heavy equipment at the site out of fear concrete and other debris might fall on anyone still alive under the rubble.
Fortin Augustin, the preacher who built the College, was charged with involuntary manslaughter late on Saturday, Garry Desrosier, a police spokesman, said.
Scores of students and teachers were trapped inside the three-storey building and neighbouring homes when it collapsed while construction work to add an extra floor was being carried out.
“He told me he built the building all by himself. He said he didn’t need an engineer because he had good knowledge of construction,” Joseph Manes Louis, a prosecutor, said.
Rene Preval, the Haitian president, has said that he believed poor construction methods, including a failure to use reinforced steel, had been to blame for the tragedy.
Preval and Michele Pierre-Louis, the prime minister, oversaw the emergency effort as recuers worked around the clock to remove people from the rubble.
|Rescuers worked through the night in an attempt to find survivors [AFP]|
The disaster occurred as classes were being held at the church school, where up to 700 students aged between three and 20 attended classes.
Local authorities had used their bare hands to pull bleeding students from the rubble before heavy equipment and international teams arrived late on Friday.
Crowds of screaming and crying parents searched for their children in the ruins, and roads around the school were so jammed with people that some rescuers had to be brought in by helicopter.
Haiti, the poorest country in the Americas, is still recovering from four tropical storms and hurricanes that killed more than 800 people and destroyed 60 per cent of its crops in August and September.