The summit aims to promote co-operation between the two neighbours, but has been overshadowed by US concerns about Pakistan's stability.
Pro-Taliban fighters have taken control of several areas of North West Frontier Province, pushing to within 100km of the capital, after a controversial truce in the Swat valley collapsed.
"The president is deeply concerned about the security situation," Robert Gibbs, the White House press secretary, said.
"That's why we're sending additional troops to Afghanistan, and that's why we'll talk with both the Afghans and the Pakistanis about our renewed commitment in helping them seek the aid that they need to address those extremists."
Richard Holbrooke, the US special envoy to Pakistan and Afghanistan, insisted that Pakistan was not a "failed state" and said Washington was committed to supporting the government in Islamabad.
"Pakistan's of such immense importance to the United States, strategically and politically, that our goal must be unambiguously to support and help stabilise a democratic Pakistan headed by its elected president, Asif Ali Zardari," he told congress.
"This war against terrorism will succeed only if we fight it from a higher platform of morality"
Holbrooke also warned that Pakistan must start to "show results" in its efforts to address the security situation along the border with Afghanistan.
"Pakistan must demonstrate its commitment to rooting out al-Qaeda and the violent extremists within its borders," he said.
Zardari held a 90-minute meeting with members of the foreign affairs committee of the US House of Representatives in an attempt to reassure them that Islamabad was tackling the violence along the porous border with Afghanistan.
Relations between Islamabad and Kabul have been strained over the security situation, with Afghan officials accusing elements within Pakistan's military and intelligence services of supporting the violence and doing little to stop fighters crossing the border.
But under Zardari, Pakistan and Afghanistan have increased their co-operation on issues of mutual concern.
Karzai told a Washington-based think-tank on Tuesday that the two nations needed to address the havens for Taliban and al-Qaeda fighters inside Pakistan's border regions.
|The US says it will send 20,000 more troops to Afghanistan to tackle rising violence [AFP]
"The return of the Taliban is because we did not address the question of sanctuaries in time. Unfortunately, today, Pakistan is suffering with us massively as a consequence of that," he said.
Afghanistan's relationship with the Obama administration has also been tense after Washington voiced its frustration at Kabul's apparent ineffectiveness and corruption.
However, Karzai told the Brookings Institution that ties were "very, very strong".
Meetings between the three delegations begin on Wednesday morning at the state department and continue in the afternoon with heads of government sessions at the White House.
Obama will hold separate meetings with each of the leaders as well as a joint session.
Zardari and Karzai are expected to raise their concerns about the deaths of civilians in US-led and Nato military operations, warning that they turn the public against the foreign forces and the government.
"This war against terrorism will succeed only if we fight it from a higher platform of morality," Karzai said. "Money can't buy you love, as you say it in America, no matter how much it is."
Local officials said on Tuesday that scores of civilians died in US air raids in Afghanistan's Farah province. The US military said a joint investigation had been launched with the Afghan government.