"I could only hear the shouting of the people."
Al Jazeera's Imran Khan, reporting from the scene, said four armed men were killed on the scene, but four others had managed to escape.
He said the military had secured the area and that at least one helicopter was being used to search for the attackers.
"The city itself has been opened up. The streets surrounding the mosque, however, are still under heavy security. The army have mounted a very large presence here."
There was no immediate claim of responsibility, but our correspondent said the attack was significant because it may have caused the deaths of some "very senior military officers".
"Whoever was responsible for this attack will be saying that this is a coup for them because in previous attacks on security installations what normally happens is its low-level officers that form the brunt of [the attack]," he said.
"We are hearing two two-star generals have been killed in [Friday's] attack, as well as a brigadier and a major-general. Those are unconfirmed at the moment, but if that's true, this is the higher echelon of the military.
"The big question is how they were able to get inside with machine guns and .... penetrate such high security."
The attack appeared to be the latest in a series to hit Pakistan in recent months as the military presses ahead with an offensive against al-Qaeda and Taliban fighters in the country's northwest.
'Reaction to offensive'
Major-general Athar Abbas, a spokesman for the Pakistani military, confirmed that six high-ranking officers were among the dead and said authorities have yet to establish how the security breach occurred.
But he said it was a case of Taliban and al-Qaeda fighters reacting to the army offensive against them.
"We have yet to establish who the attackers are, but apparently on the face of it they belong to the same organisation against whom the army is operating in the area of South Waziristan," he said, referring to the Pakistani Taliban.
"The military has swept the area - their strongholds and hideouts - and [the fighters] are on the run.
"But they have their agents in the cities and the towns, so ... they will conduct these acts. But since their bases have been destroyed they have been disconnected."
Two of the biggest recent attacks in the country have taken place in Rawalpindi, including a suicide bombing attack that killed 35 people on November 2.
A raid on the army headquarters on October 10 led to a day-long standoff that left 23 people dead, including nine attackers.