The blast came hours after the regional emergency ministry branch warned residents that it was carrying out "planned detonation of unexploded shells" from the arms depot, promising that it was following "safety measures".
Military investigators were sent to the city to establish the reason for Monday's blast, the regional military prosecutors told the Interfax news agency.
Al Jazeera's Neave Barker, reporting from Moscow, said: "With this the second blast in as many weeks, there are going to be some serious questions asked in the upper echelons of the Kremlin, I imagine."
Russia's poorly trained and badly paid military has suffered a number of accidents with their ageing equipment in recent years.
"These kinds of incidents have been happening sporadically over the last few years, but this is the third such incident in the last two months," Barker said.
"Russia's armed forces haven't been significantly modernised since the Soviet Union, although President Medvedev called for a complete overhaul of the armed forces at the beginning of the year.
"These changes, these attempts to modernise Russia's armed forces have been somewhat slow and quite clearly there a number of ageing installations and bases across the country.
"This incident this afternoon can be directly attributed to the fact that Russia's armed forces need a major cash injection."
Dmitry Medvedev, the Russian president, was scheduled to visit Ulyanovsk on Tuesday to meet Ilham Aliyev, his Azeri counterpart.