Helen Olafsdottir, a spokesperson for the Sri Lankan Nordic Monitoring Mission, told Al Jazeera on Wednesday: "We have received reports that 50 people have been killed and at least 600 injured by a rocket which landed on a building where internally-displaced people were squatting.
"We have a team on the ground now helping to pull people out of the rubble. It is real chaos."
Olafsdottir said thousands of people were trapped inside.
The incident occured in the town of Kathiraveli in the eastern district of Batticaloa.The Liberation Tigers of Tamil Eelam (LTTE) had earlier reported that army fire had struck the camp and killed at least 45 people and injured 125.
The Sri Lankan army said it had no details of casualties.
Amnesty International, the international rights group, condemned the incident.
Purna Sen, Amnesty's Asia Pacific director, said: "It is appalling that the military should attack a camp for displaced people - these are civilians who have already been forced from their homes because of the conflict."
'Human shields' denial
However, the military denied it had attacked a refugee camp and said it had retaliated with artillery and mortar bomb fire after the LTTE fired on military camps in the area during the early morning, injuring five soldiers.
"We located the LTTE gun positions with radar and launched a precise retaliatory attack. It was not a refugees camp, it was an LTTE camp," Prasad Samarasinghe, a spokesperson for the Media Centre for National Security, told Al Jazeera.
Samarasinghe said the Sri Lankan army had been pressing civilians to leave Tiger areas but the LTTE had been using them as human shields.
The Tigers denied the accusation and condemned the incident in an internet statement.
"The mindless and cruel attack on a helpless refugee population which has already been subjected to blockades of all sorts of essential items is difficult to understand," the fighters said.
Peace talks collapse
The LTTE and the Sri Lankan military have clashed repeatedly in the north and the east in recent months, since peace talks collapsed in late October.The first peace talks in eight months failed over a Tiger demand that the government reopen the main north-south highway which runs through Tiger territory to Jaffna, a government-held enclave on the northern tip of the island.
Tiger areas have been bombed repeatedly since peace talks failed.
More than 65,000 people have been killed in the conflict since 1983.