In course of his short visit, the US president met Hamid Karzai, his Afghan counterpart, on Wednesday and praised the progress made since the fall of the Taliban.
"We are impressed by the progress that your country is making Mr President, a lot of it has to do with your leadership," Bush told Karzai at a joint press conference.
He spent just over four hours on the ground during his surprise visit. He later flew to New Delhi, India, where tens of thousands of Indians demonstrated on Wednesday against his visit. He would visit Pakistan later in the week.
"People all over the world are watching the experience here in Afghanistan. I hope the people of Afghanistan understand that democracy takes hold," he said.
"We are impressed by the progress that your country is making Mr President, a lot of it has to do with your leadership"
"You are inspiring others and that inspiration will cause others
to demand their freedom, and as the world becomes more free the world will become more peaceful."
Bush said the US embassy in Kabul, which he was officially
opening during his visit, was a "clear statement" to the people of Afghanistan that the United States was dedicated to helping the country.
"It's our country's pleasure and honour to be involved in the future of this country," he added.
"We like stories of young girls going to school for the first time so they can realise their potential. We appreciate the free press. We are enthralled when we see an entrepreuneurial class grow up where people are able to grow up and realise their dreams," he said.
Bush pledged that Osama bin Laden, the al-Qaida leader, and
other planners of the September 11 terror attacks on the United States would be caught.
Bush addressed US troops during
his visit to Afghanistan
"It's not a matter of if they're captured and brought to justice, it's when they're brought to justice," Bush said.
Before leaving Afghanistan, Bush gave a pep talk to US troops. Speaking to about 500 soldiers, the US president expressed resolve a the US mission in Afghanistan.
"I assure you this government of yours will not blink, we will not yield...the United States is not cut and run."
There is an 18,000-strong US-led force stationed in Afghanistan, along with about 9000 Nato-led peacekeepers.
Bush's visit to Afghanistan came at a time when the country is still troubled by a stubborn Taliban insurgency that has claimed 1500 lives since the start of last year, including dozens of US soldiers.
The Taliban meanwhile reacted to the Bush visit, saying the secret visit showed the group had a strong control over Afghanistan.
Mullah Abdullah Akhund, the Taliban deputy leader and former defence minsiter, said: "If the American president's visit had been announced in advance, the Taliban Mujahideen would have greeted him with rockets and attacks. But Bush proved his cowardice by coming on a secret visit as a thief."
US-led forces had overthrown the Taliban government for refusing to hand over leaders of the al-Qaida network responsible for the 9/11 attacks on the United States.