"Too often the whole of these activities has added up to less than the sum of the parts," he said.
Gates' remarks came amid anger in Afghanistan over claims that a US military raid in the eastern Afghanistan province of Laghman at the weekend killed least 16 civilians.
The US military has denied the claims and said that 15 Taliban fighters died in the operation.
Gates also said that the Pentagon's efforts in Afghanistan to combat the insurgency following the overthrow of the Taliban in 2001, would in the future include better counter-insurgency training for US and international forces and efforts to improve "civil-military" co-ordination for reconstruction efforts.
"Our primary goal is to prevent Afghanistan from being used as a base for terrorists and extremists to attack the United States and our allies. And whatever else we
need to do flows from that objective," he told lawmakers.
"We can attain what I believe should be among our strategic objectives: An Afghan people who do not provide a safe haven for al-Qaeda, reject the rule of the Taliban and support the legitimate government that they elected and in which they have a stake," he said.
The top US commander in Afghanistan has requested 30,000 more troops for the country - including four combat brigades, each with about 3,500 soldiers - to help combat the Taliban.
Obama has vowed to shift military resources from Iraq to Afghanistan and Pakistan, both which he says form the central front in the struggle against "terrorism".
Gates also praised Pakistan as "a friend and partner" and said any solution to the Afghan conflict required strong relations with Pakistan given the porous border between the two nations.
Last week he appointed Richard Holbrooke, a veteran US diplomat, as special envoy to both countries.
|Gates predicted "a long and difficult fight" for the US in Afghanistan [AFP]
Gates said that although violence remained low in Iraq there was "still the potential" for setbacks.
He described the prospect of a stable Iraq as "crucial to the Middle East", but warned against looking for a swift resolution to the conflict.
"As our military presence decreases over time, we should still expect to be involved in Iraq on some level for some many years to come," he said.
US forces are currently due to withdraw from major Iraqi cities by the end of June this year and from the country by 2012, as part of a US-Iraq agreement.
The US currently has about 34,000 troops in Afghanistan and around 142,000 troops in Iraq.