Bainimarama said: "We have reasonable grounds to believe that the life of the state is being threatened. Should we be pushed to use force let me state that we will do so very quickly. The military will suppress very quickly any uprising against us."
Bainimarama, who staged a bloodless coup on Tuesday after a year-long power struggle with Qarase, ordered a security cordon to be set up around Suva.
Soldiers also rounded up opponents, including Pita Nacuva, the speaker of parliament, and Moses Driver, the acting police commissioner, and took them to the country's main military barracks in Suva.
Intervention in parliament
Checkpoints were established around the city and the coup chief also recalled all military reserves to barracks to support the state of emergency, the Legend radio network reported.
Senators in the Fijian parliament moved a motion on Wednesday condemning the coup, but before they could vote, the sitting was interrupted by soldiers who took over the building, and the senate was adjourned.
Qarase, who was forcibly flown to his home island of Vanua Balavu with his wife before sunrise on Wednesday, called on Fijians to stand up for democracy, but to do it peacefully.
Various countries have condemned the coup - the fourth in 20 years. Australia and New Zealand imposed sanctions on Fiji's military and the US suspended aid.
Bainimarama accused Qarase of being too soft on those behind Fiji's last coup in 2000. He has appointed a caretaker prime minister and said an interim administration would prepare Fiji for fresh elections, but has given no timetable.