The gap between the Dutch-speaking Flemish majority and the French-speaking minority is so wide that the premier suggested that the end of Belgium as a country was looming.
In an unusual declaration on Monday, the prime minister said that Belgium's constitutional crisis stems from the fact that "consensus politics'' across Belgium's widening linguistic divide no longer works.
"The federal consensus-model has reached its limits," Leterme said.
He failed to get his cabinet - an alliance of Christian Democrats, Liberals, Socialists and nationalist hardliners from both language camps that took office on March 20 - to agree on a future together by devolving more federal powers to the Dutch-speaking Flanders region and Francophone Wallonia.
Francophone parties had expressed surprise at Leterme's resignation.
Didier Reynders, the vice-premier urged him to stay on, saying the government must
go ahead with its social and economic programme.
Elio di Rupo, leader of the Francophone Socialists, said the constitutional reform negotiations were held in a "constructive, positive climate".
But mainstream Flemish parties, including Leterme's own Christian Democrats, accused French-speaking parties of foot-dragging and not negotiating in good faith.
Granting Belgium's Dutch and French-speaking communities more self-rule began, gradually, in the 1970s, in such areas as culture, youth affairs and sports.
Since then education, housing, trade, tourism, agriculture and other areas were shifted from the federal government and Flanders, Wallonia and bilingual Brussels were given regional governments and parliaments.
Now Francophone parties accuse Dutch-speakers of trying to separate themselves completely from French-speaking Wallonia, where the 15 per cent unemployment rate is triple that of Dutch-speaking Flanders.
Flemish parties want their more prosperous, Dutch-speaking northern half of the country to be more autonomous by shifting corporate and other taxes, some social security measures, transport, health, labour and justice matters to the language regions.
Mainstream Flemish politicians say there is room for more regional autonomy in one country but hardline nationalist parties in Flanders advocate the breakup of Belgium.
Key among them is the Flemish Interest party, which received 20 per cent of the Flemish vote in 2007.