But Abdul Shakoor, a refugee, said: "They started demolishing our houses without any reason. When we resisted, they opened fire on us."
Ghilzai said two people were killed in the attack while a doctor running a private clinic near the camp said he had seen three bodies.
Pir Alizai is one of two camps Pakistani authorities plan to shut down by June 15 and two more will be closed later this year.
'Death to Pakistan'
Thousands of Afghans also protested outside the Pakistani embassy in Kabul, chanting "Death to Pakistan, death to Musharraf", after the bloodiest clash in decades on the disputed border last weekend.
Thirteen Afghans were killed in the fighting, including two schoolchildren, police said.
Wednesday's protests, part of a series held against Pakistan in several areas of the country in recent days, ended peacefully, residents said.
Police said security was tight outside the embassy and there was no violence.
Protesters have stormed Pakistan's missions in Kabul several times in the past.
"We will teach you a lesson, Pakistan," shouted Hazrat Mir Zazai, joining other protesters yelling anti-Pakistan slogans on loudspeakers.
Relations between the neighbours, both US allies, have deteriorated in the past 18 months as attacks in Afghanistan have increased.
Both sides blame the other for a resurgence of Taliban militants.
The foreign ministry issued a strongly worded statement on Monday urging "Pakistan to prevent such action that would entail dangerous consequences for the stability of the region".