Bainimarama, who staged a bloodless coup on Tuesday after a year-long power struggle with Qarase, ordered a security cordon to be set up around Suva.
Soldiers also rounded up opponents, including Pita Nacuva, the speaker of parliament, and Moses Driver, the acting police commissioner, and took them to the country's main military barracks in Suva.
Check points were established around the city and the coup chief also recalled all military reserves to barracks to support the state of emergency, the Legend radio network reported.
Senators in the Fijian parliament moved a motion on Wednesday condemning the coup but before they could vote the sitting was interrupted by soldiers who took over the building, and the senate adjourned.
Qarase, who was forcibly flown to his home island Vanua Balavu together with his wife before sunrise on Wednesday, called on Fijians to stand up for democracy, but to do it peacefully.
The international community has condemned the coup – the fourth in 20 years.
Australia and New Zealand imposed sanctions on Fiji's military and the United States suspended aid.
Bainimarama accused Qarase of being too soft on those behind Fiji's last coup in 2000. He has appointed a caretaker prime minister and said an interim administration would prepare Fiji for fresh elections, but has given no timetable.