Tariq Azim Khan, deputy information minister, said General Musharraf was safely in his office some 2km away at the time of the blast.
A Reuters journalist saw body parts on the road near a perimeter wall of the residence of the chairman of the joint chiefs of staff, General Tariq Majid.
Earlier, Al Jazeera's correspondent Nadim Baba, speaking from Islamabad, described the blast as very powerful and said two of the dead were policemen.
"We know the blast site was an extremely sensitive place," he reported.
"A government minister has admitted that President Musharraf was in the building at the time. He was meeting local governors and chief ministers to discuss security in various parts of the country."
Musharraf has survived at least three assassination attempts - two in December 2003, and one in July as his aircraft took off from Rawalpindi's airport.
Suicide and roadside bomb attacks on security forces have multiplied since commandos stormed the Red Mosque in the capital, Islamabad, in July to crush a Taliban-style movement.
More than 100 people were killed in the fighting.
The security situation in the country has continued to deteriorate, and scores of people have been killed in fighting between security forces and anti-government fighters in the scenic valley of Swat in North West Frontier Province during the past few days.
The worsening security comes at a time of intense political uncertainty in Pakistan.
An attack, possibly two suicide bombers, killed 139 people at a procession in the southern city of Karachi held to welcome the return of Benazir Bhutto, a former prime minister, from self-imposed exile on October 18.
Source: Al Jazeera and agencies