Mingora is the main town in Swat.
"How can we follow the agreement with them?" Khan said.
Talks over the implementation of the deal started to falter and last month Sufi Muhammad, a regional religious leader who had brokered the deal, walked away from negotiations, saying the government had been dragging its feet.
Khan said the Pakistani Taliban would continue to implement the sharia "as soon as possible".
Escape from fighting
Thousands of people were told by authorities on Tuesday to flee the region. The next day residents defied a curfew to escape fighting in the region.
Deadly clashes took place on Wednesday in Mingora, where local officials said Taliban fighters had defied curfews to occupy government buildings.
Gunfire was heard in Mingora, although access by the media was limited. Witness said the Taliban, identified by their black turbans, were on most streets and taking up positions on high buildings.
Fighters were also reported to have surrounded a paramilitary base at a power station in the town.
Kamal Hyder, Al Jazeera's Islamabad correspondent, said: "Malakand division is under curfew, many people are stuck and cannot get out to safety.
"We have been getting reports that some of the thousands defying the curfew have been fired upon, there have been a high number of casualties of people leaving.
"We saw a military convoy going towards Mingora carrying reinforcements in what appears to be the prelude to the third phase of the operation against Swat.
Describing the fighting, our correspondent said: "This is a major catastrophe unfolding in the North West Frontier Province.
"People coming out of Swat are saying that their leaders have sold them to foreign powers. Very strong, emotional language from people who are running for their lives."
In his interview to Al Jazeera, Khan, the Taliban spokesman, said that the group's fighters were in control of "90 per cent" of the Swat valley.
He hinted that aid money from the US to the government of Asif Ali Zardari, the president, was influencing the leadership.
|People have had to leave with just what they can carry [AFP]
"Zardari just needs money. They [the US and Pakistani government] want to kill the nation. They are never thinking about the nation, the country and the Muslims," he said.
Those comments came as Zardari prepareed to meet his counterparts from Afghanistan and the US on Wednesday to discuss how to handle the conflicts in the region.
Hundreds of thousands of people have fled fighting between the Pakistan military and the Taliban in different parts of the northwest since last August.
The exodus has put another burden on Pakistan's economy, already being propped up by a $7.6bn loan from the International Monetary Fund.
Zardari's government has said that up to half a million civilians could be displaced by the current fighting.
In a move to address the humanitarian crisis, a camp has been set up for the displaced in Dargai, a town adjoining Swat.
Mian Iftikhar Hussain, the NWFP information minister, said: "In view of the situation in Swat, at least 500,000 people can migrate from that area. Camps are being established for them."
Khushal Khan, a senior administrator in Swat, accused Taliban fighters of laying mines, making the civilian escape highly risky.