However, Hamidou Hima rejected the characterisation of Tandja's removal as a coup.
"We left the political actors to try and find a solution. This did not happen. Social tensions got worse. We didn't launch a coup, we just reimposed legitimacy, because this had already disappeared," he said.
Tandja 'in good conditions'
Tandja has not been seen since soldiers stormed a cabinet meeting on Thursday, but Hamidou Hima said the ousted president was in custody at the presidential palace.
"Mr Tandja is in a service quarters of the presidency and is being kept in very good conditions," he said.
Hima said that three of Tandja's ministers who were with the president at the time of the coup were also still being held.
The African Union and Ecowas both condemned the overthrow of Tandja, along with the United Nations, European Union and former colonial power France.
But after meeting members of the military, the delegate from Ecowas spoke positively of their intentions.
"They have given us the necessary guarantees and all this will be done with the participation of civil society and the political parties," Mohamed Ibn Chambas, the head of the 15-nation Ecowas, said.
"Dialogue will be opened with all the vital forces of the nation which will end in the drawing up of a new constitution and a period of transition.
"We were encouraged by the fact that the authorities themselves are mindful that this is not their normal function and they are eager to finish this task and go back to their normal military and security duties."
The overthrow of Tandja appears to have been met positively inside Niger, with thousands of people turning out on the streets to back the military over the last two days.
Voix du Sahel radio said that thousands of people had taken part in "gigantic demonstration" in Zinder, the country's second city, on Sunday.
The turnout was "to salute the defence and security forces for the patriotic work which it has accomplished," the radio station reported.
That came after about 10,000 people gathered outside the parliament building in Niamey, the Nigerien capital, the previous day after a coalition of opposition parties, trade unions and human rights groups called on people to show their support for the military.
The military rulers have suspended the constitution have suspended the constitution brought in by Tandja after he won a referendum that gave him three extra years in power.