District forest official Mohammed Ali Kabir said a herd of elephants entered the Balukhali camp in Ukhiya town early on Saturday and trampled tents where several refugees were sleeping.
Kabir said that four other people were injured in the attack and many others fled to safety.
At the refugee camp, cooking utensils the family had brought from Myanmar lay crushed under a jumble of bamboo sticks and clothes.
Many trees on the forested hills of Balukhali in southern Bangladesh have been chopped down to house the massive influx of ethnic Rohingya escaping violence in neighbouring Myanmar.
Tarpaulin and bamboo shelters have been built on elephant walkways in some areas, sparking environmental concerns, as the country struggles to accommodate an unprecedented number of people.
It was the third attack by wild elephants on the refugees in Bangladesh's Cox's Bazar district, which borders Myanmar. At least seven people died in the two previous attacks.
Abdul Sukoor, 30, whose tent on the edge of the camp was also trampled, said he and his family managed to escape when they heard the screams of other refugees as the elephants approached.
He is now moving to a tent further inside the camp, but said he was still worried. "We have to be constantly alert at night," he said.
More than 530,000 refugees from Myanmar have fled to Bangladesh over the last two months since attacks by Rohingya militia on security posts triggered a Myanmar army operation that the United Nations has described as ethnic cleansing.
Source: News agencies