Musharraf said the neighbours must overcome mutual mistrust and concentrate on this "critical" endeavour".
He said: "There is no other option for both countries other than peace and unity, trust and co-operation.
"There is no justification for resorting to terrorism."
Before Musharraf's closing address, a joint declaration was distributed to the delegates that summarised the outcome of the jirga.
It called for measures against what it sees as terrorism, including rooting out training grounds, and for the establishment of a council to push for closer co-operation between the two neighbours.
In his address, Musharraf said: "Our societies face a great danger (from) a small minority that create violence and backwardness.
"These forces are disrupting peace and harmony, impeding our progress and development."
Referring to the Taliban, Musharraf said: "There is no doubt Afghan militants are supported from Pakistan soil. The problem that you have in your region is because support is provided from our side."
But he indicated that even the Taliban have a place in Afghanistan.
|The peace jirga in Kabul is being attended by|
about 700 tribal elders and politicians [EPA]
He said: "Taliban are part of the Afghan society. Most of them may be ignorant and misguided, but all of them are not diehard militants and fanatics who defy even the most fundamental values of our culture and our faith."
Musharraf had been expected to open the "peace jirga" on Thursday with Karzai but pulled out at the last minute citing security concerns.
He reversed his decision after phone calls from Karzai and Condoleezza Rice, the US secretary of state.
Kamal Hyder, Al Jazeera's correspondent in Islamabad, said: "A lot of people here are asking if this is too little, too late.
"Musharraf and Karzai have both become unpopular in their own countries and many people say this jirga is not going to achieve anything."
Delegates had split into committees focused on topics such as the reasons for "terrorism", the fight against drugs - said to finance fighters - and good neighbourliness, Asif Nang, a spokesman, said.
The results of these findings are expected to help the formation of a strategy, to be announced on Sunday, before Musharraf and Karzai were to close the meeting, the spokesman said.
Recommendations are likely to include the establishment of a joint commission to analyse factors fuelling terrorism and another on fighting the drugs trade and organised crime, Afghan media reported on Sunday.
Wave of violence
On the ground, meanwhile, violence continued unabated.
Taliban attacks across Afghanistan have killed 29 people, including four international soldiers and nearly two dozen fighters, military officials said on Sunday.
Three soldiers with the US-led multinational force and their Afghan interpreter were killed near the border with Pakistan when they were hit by a bomb during combat, the force said in a statement.
Taliban fighters were responsible for the attack in Nangarhar province, a spokesman said, claiming the soldiers were US nationals.
Earlier, the British defence ministry announced that a British soldier was killed and five wounded after their patrol came under fire from Taliban fighters in southern Afghanistan on Saturday.
The attack was in the volatile Sangin district of Helmand province. Fighters also ambushed an Afghan army patrol in Sangin overnight, the Afghan ministry of defence said.
The attack sparked a fierce gun battle in which seven fighters were killed and seven wounded, it said.
Fighter jets were called in to attack ground targets after the Taliban stormed an Afghan army post in southern Uruzgan province on Saturday.