Around 800 people were reported to have attended the ceremony, including the chiefs of the different branches of the military, the governors of the four provinces, and the chief ministers for each of those provinces.
"Let it be known to all and sundry that Asif Ali Zardari has on this ninth day of September taken the oath as president of Pakistan," an official proclamation read out at the ceremony, said.
'Will and vision'
The two presidents appeared together at a news conference shortly after the inauguration, vowing to fight "terrorism" together.
At the news conference, Zardari dedicated his presidency to his wife, saying his election provided democratic impetus to the government's fight against terrorism.
"The government of Pakistan already has a comprehensive plan. Yesterday's war may not have had the people behind it, but today's war has the people behind it, it has the Pakistani president behind it, who himself is a victim of terrorism," Zardari said.
Karzai had warm words for his counterpart, praising the Pakistani leader's "good will and a vision".
"I find the government of Pakistan, in the form of its president and prime minister, well intentioned," he said.
"[They have] an immense will to bring wealth to the people of Pakistan ... and that intention can only come with the defeat of terrorism."
Zardari, co-chairman of the Pakistan Peoples Party (PPP), rose swiftly to power after the assassination of his wife, Benazir Bhutto, at her own election rally last year.
He was chosen as head of the PPP, co-chairing with his son Bilawal, only three days after Bhutto's death.
On Saturday, he swept the vote by members of the two-chamber parliament and four provincial assemblies to win the presidency.
Zardari's party heads a fragile coalition government which, although still in office, recently lost the backing of a key coalition party.
As president, he takes charge of a country that has been riven by violence, with nearly 1,200 people killed in bombings and suicide attacks in the past year.
The economy is also in trouble, with rampant inflation and a plunging stock market that has lost around 40 per cent of its value since January.