Initial findings gave no indication the explosion was anything other than an accident, an official said.
The explosives were stored in the basement of a four-storey building that was reduced to rubble in the residential al-Masbah area of Taiz.
Witnesses told the German Press Agency DPA that the pre-dawn explosion left eight families homeless.
They said the warehouse was owned by an arms dealer and was used for road construction in the mountainous Taiz province.
"The explosion tore apart three buildings," Muhammad Saeed, a witness said by phone. "Many people are still under the wreckage."
Security officials said search-and-rescue operations were continuing.