He heads the Republican party in a coalition government with Berisha.
Explosions at the site, where Communist-era munitions were dismantled, lasted for more than 14 hours.
The initial blasts were so large they were heard in neighbouring Macedonia.
Berisha had likened them to "a nuclear blast ... without the radioactivity."
The Albanian authorities have not ruled out foul play. Prosecutor Ina Rama said: "We are following several lines of inquiry."
Investigators will look at the security measures put in place by Southern Ammunition Company, a US contractor in charge of carrying out the dismantling of the munitions.
Preliminary inquiries indicated that the team had chosen the wrong site to dismantle the arms at the Gradec depot.
Hundreds of workers have been dismantling the munitions at various sites near Rinas, close to both the airport and a motorway, 12km north of Tirana.
Human rights groups and witnesses have said that most of the workers are badly-paid women, without training or employment contracts.
A third day of rescue work to look for those still missing and safeguard the area continued on Monday.
Rescue attempts have been hampered due to the fear of further unexploded shells being triggered.
Rescue workers have found at least five bodies near the site of the blast, but could not identify them as the corpses had been almost fully carbonised.
At least 10 people are still considered missing, the government-published list showed.
Hundreds of people, mostly relatives of the victims or villagers from the region have blocked the main road in the area, demanding that the authorities speed up their search.
Some of them tried to break through the military cordons securing the area, in a bid to reach their homes and search for the relatives themselves.
About 1,000 soldiers and 500 police had been involved in the rescue operation.
Destroying the old munitions is one of the conditions Albania must fulfil to become a member of the North Atlantic Treaty Organisation (Nato).
Before resigning, Mediu had said there were still about 100,000 tonnes of antiquated munitions in the country posing a threat to the population.
Along with Croatia and Macedonia, Albania hopes to be invited to join Nato at a summit in Bucharest early next month.
Gerdec has been declared an emergency zone, and Berisha has promised relief for villagers who lost their homes.