Mohammed was arrested in early 2002 as he returned to Pakistan from Afghanistan and jailed on a weapons charge.
Faridullah Khan, the provincial government spokesman, said that the peace pact was signed by Mohammed's deputy and eight other religious leaders, along with four officials, including three provincial government ministers.
Under the six-point agreement, which covers Swat and neighbouring districts, Mohammed's group renounces attacks on the government but can peacefully seek the implementation of the sharia.
However, his son-in-law Fazlullah took over as leader of Tehrik-e-Nifaz-e-Shariat Mohammadi (Movement for Implementation of Muhammad's Sharia Law) and over the past year he has led attacks on government forces.
Last year, a major military operation was launched in the NWFP's Swat valley to take on the group which was exercising considerable control over the region.
It was not immediately clear if Fazlullah had agreed to lay down arms as a result of the deal.
Mohammed, who is believed to be in his 70s, has spent the past five months in a hospital in Peshawar due to ill health.
He left the hospital under police escort, accompanied by followers who shielded his face, apparently to prevent him being photographed. He later met the province's chief minister.
About 30 other members of the group were also released as part of the agreement.
Al Jazeera's Kamal Hyder in Islamabad said there was a growing demand for political reconciliation to sort out Pakistan's problems.
"The new government has announced that it wil give reconciliation a chance, and the release of Sufi Mohammed is a step in that direction," he said.
The federal government and the administration in NWFP wants to use a combination of dialogue and development to curb the violence that many Pakistanis blame on the support given by Pervez Musharraf, the Pakistan president, to the so-called war on terrorism.
Britain, a major provider of development aid to Pakistan, backed the policy after meeting the president and Yusuf Raza Gilani, the prime minister.
David Miliband, Britain's foreign minister, said that he supported "reconciliation with those who are willing to reconcile".