Pierre Nkurunziza, the Burundian president, and Agathon Rwasa, leader of the Hutu Forces for National Liberation (FNL), were joined by regional and international leaders at the signing of the agreement on Sunday.
Thabo Mbeki, the South African president, told both sides: "An important agreement has been made to stop hostilities and reach a comprehensive agreement in two weeks. I believe it will not take us longer than two weeks.
"Let's stop these guns."
South Africa is the mediator in the talks, and Tanzania has worked for more than a year to bring Rwasa and his leadership to negotiations.
Originally, the two sides had been due to sign a final pact on Saturday, but talks went late into the night as assembled dignitaries pressured both sides to reach a deal.
During the talks, the rebels twice shelled the capital, Bujumbura, killing one and wounding at least 15 others.
The Burundian army regularly hunts the FNL in their bush hideouts with attack helicopters.
A pact with the FNL is seen as one of the final hurdles to stability in a nation recovering from more than a decade of civil war pitting the Hutu majority against the politically and economically dominant Tutsi minority.
At least 300,000 people were killed in a series of ethnic reprisals sparked by the 1993 assassination of Melchior Ndadaye, the first elected Hutu president, by Tutsi paratroopers.
Earlier talks failed to produce a deal, although Tanzania brokered a May 2005 cease-fire which was broken within days.
Burundi, a coffee-growing nation of seven million on the shores of Lake Tanganyika, has been lauded as a model on the continent because of its progress and relative stability in following a UN-backed peace plan drawn up by regional leaders.