No group has claimed responsibility for the explosion and security sources have not accused any group.
Shibam is a Unesco world heritage site and dubbed the "Manhattan of the Desert" because of its high-rise mud-brick buildings.
"Maybe it was a terrorist attack, but it could also be remnants of dynamite from a mine going off," another official said.
Khaled al-Hammadi, a journalist in Yemen's capital, Sanaa, told Al Jazeera: "This attack comes after two others in the last two years. One on Belgian tourists, and another which attacked a Polish camp.
"These attacks came after the announcement of new leadership for al-Qaeda in the Arabian peninsula.
"The Yemeni government have conducted a lot of operations to combat al-Qaeda's operations in the country with the co-operation of the international community, particularly the US government.
"There are currently a lot of [court] trials in which al-Qaeda members are accused," he said.
In January 2008, two Belgian tourists were killed in the attack.
The Arabian peninsula country, one of the poorest outside Africa, has been seen as a stronghold for al-Qaeda-linked fighters.