Gates told a news conference: "After the Soviets left, the US made a mistake. We neglected Afghanistan and extremists took control of this country - and the United States had the result of that on September 11, 2001.
"We will not make that mistake again. We are in it for the long term."
The Taliban rose to power after the Soviet withdrawal, but were ousted by US-led forces for sheltering al-Qaeda leader Osama Bin Laden, blamed for the attacks on New York and Washington which killed nearly 3,000 people.
Pakistan has come under pressure from its allies to crack down on fighters crossing the border into Afghanistan amid a renewed Taliban campaign that claimed 4,000 lives last year, mostly those of rebels.
The defence secretary, accompanied by senior military officials, met Musharraf at his presidential office in the garrison city of Rawalpindi, outside Islamabad.
Gates said after the talks that Pakistan and the US had a "mutual interest in improving our effectiveness and improving our co-operation and understanding so that we have a real opportunity this spring".
He said Musharraf had been meeting Pakistani army commanders to see how they could improve their operations on the porous Afghan border.
"We discussed the coming spring military activity on the border and the measures that the Afghans and the Nato alliance and the US and Pakistan are working on together," he said.
|"We discussed the coming spring military activity on the border and the measures that the Afghans and the Nato alliance and the US and Pakistan are working on together"|
US defence secretary
Gates said he had thanked Musharraf "for Pakistan's help in the war on terror". He said: "Pakistan is clearly a very strong ally of the United States on this."
The US defence secretary said the US was also grateful to Musharraf for cracking down on extremism in his own country, which has been rocked by a recent wave of attacks blamed on pro-Taliban elements.
Gates arrived in Pakistan from Europe, where he appealed to Washington's allies to contribute more troops, equipment, and support personnel for the 35,000-strong Nato-led force in Afghanistan.
US military officials have claimed bitterly that Pakistani border forces were turning a blind eye to the cross-border movements of fighters, fuelling the resurgence of violence in Afghanistan.
But Musharraf recently said security along the Pakistan-Afghanistan border was the joint responsibility of all the forces nearby, and Pakistan could not win the fight against militancy on its own.