The explosion hit the centre of the city, close to the US embassy, a spokesman for the international peacekeeping force which patrols Kabul said. No one was reported injured.
Kabul deputy police chief Matahullah Rahmani said the explosion at around 6.10 pm (1340 GMT) was probably caused by a rocket, but that he could not be sure.
US embassy staff were not immediately available to confirm Rumsfeld had left following his meetings with Karzai and two Afghan regional commanders in the northern city of Mazar-i-Sharif.
Earlier Rumsfeld held talks in the main northern city of Mazar-i-Sharif with Uzbek General Abdul Rashid Dostam and his Tajik rival Muhammad Atta.
He hailed both men for agreeing to disarm and demobilise their forces after meeting at the headquarters of a British civil-military team involved in reconstruction and security reforms in the troubled area.
"We had a discussion about the fact that they have both agreed to the programme for turning in heavy weapons and the disarmament programme," Rumsfeld told reporters during the joint press conference with Karzai in Kabul.
"They have already begun walking down that path and we certainly acknowledge that and encouraged it.
"The Taliban ... will not be able to disrupt the process (towards democracy)"
"We believe that that's an important step for this country to have the heavy weapons brought in," he said.
Factional violence, particularly in northern Afghanistan, has been a major stumbling block for the country's US-backed government's efforts to exert its authority outside Kabul following the 2001 collapse of the Taliban government.
In Kabul, powerful defence minister, Muhammad Qasim Fahim, also assured Rumsfeld that he too was committed to moving his forces' heavy weapons out of Kabul and drawing down privately-aligned militia there.
While southern and eastern Afghanistan have been troubled by a wave of violence blamed on resurgent Taliban fighters, Rumsfeld and Karzai said the hardliners would not delay or disrupt elections scheduled for 2004.
"The Taliban ... will not be able to disrupt the process" towards democracy, said Karzai.
Praise for feuding rivals
In Mazar-i-Sharif Rumsfeld praised Atta and Dostam for agreeing to disarm but told reporters afterwards that the pace was uncertain.
"Each of them had initiated that process. It is underway and that's a very good thing," he said.
Karzai and Rumsfeld (R) in Kabul, just before the blast
"At what pace it would proceed, I guess remains to be seen," he said.
Rumsfeld's trip was aimed at demonstrating support for Karzai's efforts to "stabilise the north," a senior US official travelling with the defence secretary told reporters.
"The signal we are trying to send is that this is an important effort which Karzai is trying to do to extend the reach of the central government into the provinces and particularly in the north."