At a joint news conference at the presidential palace, Gates stressed the US would not leave Afghanistan, even though the US president has announced plans for a gradual military withdrawal in 18 months.
However, the defence secretary said Karzai needed to take a tougher line on corruption, saying he would press the Afghan president to appoint "honest" ministers.
"It is important to us, in terms of all of our success, including the Afghan success, to have capable and honest ministers in the areas that matter the most to us," he said, referring to the Afghan defence and interior ministries.
"The truth of the matter is the incumbents in these jobs, as far as I'm concerned, fill that criteria."
'Too broad brush'
Gates said there was a tendency to paint the Afghan government "with too broad a brush" when there are "competent, capable, honest ministers and there are capable, competent and honest governors. And we just need to encourage that."
Playing down the need for a wholesale government shakeup, Gates described the defence and interior ministers as "capable" and effective partners.
Al Jazeera's Steve Chao, reporting from Kabul, said corruption was among the major issues discussed during Gates' visit.
"This [corruption] is a massive issue. The challenge cannot be understated," he said.
"A recent US investigation at Kabul airport found that $10mn was being smuggled out of this country every day.
"Karzai acknowledges there has to be a massive campaign to clean up corruption. A number of his cabinet ministers are [already] under investigation for corruption.
"Yesterday Kabul’s mayor was sentenced to four years in jail for squandering upwards of $16,000."
Gates' visit comes a week after Barack Obama, the US president, announced the deployment of 30,000 extra troops to Afghanistan and gradual pullout that is expected in 2011.
Gates said he would meet US troops and tell them: "We're in this thing to win."
He is the most senior US official to meet Karzai since Obama announced his revised war strategy for Afghanistan and Pakistan last week.
Karzai, whose re-election was tainted by allegations of rampant fraud in the August 20 election, pledged in his inauguration speech to name competent and honest ministers.
His cabinet is expected to be announced in the coming days.