The plan, which could also see the government negotiate with the Taliban's top echelon, will be crafted at a peace conference next month, he said.
Karzai has already extended the government's offer to members of the Taliban who renounce ties to al-Qaeda and agree to embrace the Afghan constitution.
Gates said that he supported Karzai's efforts to promote reconciliation but cautioned that a peace deal would probably only come when armed groups understood that the odds "are no longer in their favour".
Just hours after Gates toured an area of southern Afghanistan on Tuesday from where international forces recently drove out the Taliban, two Nato soldiers were killed when a suicide bomber blew himself up outside a military base, Nato and local government sources said.
Lieutenant-Commander Iain Baxter, a Nato spokesman, confirmed two alliance soldiers had been killed in a suicide attack in Khost but was unable to give further details about the incident or disclose the soldier's nationalities.
A local government source, who declined to be named, told the Reuters news agency that a man wearing a suicide vest detonated the device outside a US military base also used by the Afghan National Border Police in the Ali Sher district of Khost, close to Afghanistan's eastern border with Pakistan.
A spokesman for the Taliban, Zabiullah Mujahid, said the attacker was a member of the Afghan group and had infiltrated the border police.
Over the weekend, fighting broke out in Baghlan province between the Taliban and fighters from Hezb-i-Islami, a group led by Gulbuddin Hekmatyar, an ethnic Pashtun leader.
Hoda Abdel-Hamid, Al Jazeera's correspondent reporting from Kunduz, said that the two groups, formerly allies, appeared to be fighting a turf war.
"According to them, what sparked these violent clashes was that a group of Hezb-i-Islami were going to speak to the Taliban, who are in the same area in the mountains, and on the way there the Taliban surrounded them, took their weapons and captured them.
"But when you speak to officials, they give you a bigger picture and they'll tell you what is going on is really a turf battle," she said.
Our correspondent said: "The Taliban have slowly been coming into the area, Baghlan province, and they now have quite a presence there. Hezb-i-Islami also has a presence there which goes back a few years.
"Apparently, because it is harvesting time, both groups are trying to extend their boundaries and get more control over villages there, because they can [extract] more taxes from the villagers."
Laal Mohammad Ahmadzai, a regional police spokesman, said that 11 Hezb-i-Islami commanders and 68 of their men had defected to the government over the attacks.
Over 60 people, among them civilians, were reportedly killed in the two days of clashes.