The authorities confirmed the Taliban withdrawal, but said local Buner Taliban and some Swat Taliban were still armed and roaming the streets.
"They have gone, but left their germs here," Abdul Rasheed Khan, the senior district police officer, said.
"Now we have about 200 local Taliban who can be seen on roadsides."
Peace deal violation
The government in North West Frontier Province (NWFP) criticised the Taliban advance as a violation of the peace deal agreement, which the authorities have come under increasing Western and domestic pressure to rescind.
"We are ready for an operation, because they have undermined the treaty, they have gone back on their word to disarm," a senior security official told Reuters news agency late on Friday.
But the Taliban said they came to Buner to oppose government forces.
Mullah Zubair, a Taliban commander of the Chinglai Taliban in Swat, said: "We have peace deal with government and under that, the government assured us that they will not deploy the army and frontier constabulary troops in the Malakand division," referring to a region in the NWFP.
"But they violated the deal and called in additional troops. The troops were besieging our fighters and wanted to establish a military base," he said.
The government on Saturday deployed up to 300 extra paramilitary police to secure Buner, local police said.
"We have full control in the area," a local administration official said.
General Ashfaq Kayani, Pakistan's army chief, defended a decision by the military not to intervene as "tactical" despite US pressure.
The army "will not allow the militants to dictate terms to the government or impose their way of life on the civil society of Pakistan", he said in a statement after a meeting of high-level military commanders on Friday.
Armed Taliban fighters on Thursday set up checkpoints and occupied mosques in Buner, clearing the streets and calling on residents not to act in an "un-Islamic" way.
|Taliban fighters set up checkpoints and occupied mosques in Buner [AFP]
Fighters clashed with district forces, leaving one policeman dead.
The beginning of the withdrawal late on Friday evening came after long negotiations between the Taliban and local authorities.
Sufi Mohamed, the Pakistani Taliban leader who mediated the initial agreement between the Pakistani government and the Taliban, arbitrated between the two parties to resolve the situation in Buner, Hoda Abdel Hamid, Al Jazeera's correspondent in Islamabad, said.
Critics who attacked the Swat deal on the grounds that government "capitulation" would only embolden the Taliban, have said that the fighters' entry into Buner has vindicated their fears.