Among the demands of the BDR, the country's border security force, is higher wages.
The mutiny, which began on Wednesday, has left a reported 50 people dead and appears to have spread to other BDR barracks across the country.
Sporadic clashes were reported to have broken out at around a dozen guard posts, including in Chittagong, the second largest city.
Al Jazeera's Nicolas Haque, reporting from Dhaka, said some of the mutineers at the Rifles' heaquarters had begun to hand over their arms to senior BDR paramilitary forces, although the situation remained volatile.
"The army [is] still creeping and moving towards the compound, just 2-3 km away from the compound are tanks that have moved inside the city. [The army] has been asking civilians to leave their homes around the compound," Haque said.
"The situation is very volatile, and the scene could change any minute."
The home minister, meanwhile, entered the compound to negotiate with the Rifle members.
About 12,000 soldiers and police were reported to have surrounding the Rifles' headquarters, while heavy artillery positioned outside.
Cellphone providers shut down mobile phone networks across the country on Thursday, an official said.
The government had issued an ultimatum to the BDR soldiers to surrender their arms by 2pm (8:00GMT) on Thursday, but the deadline passed with no surrender.
The director-general of the BDR, Major General Shakil Ahmed, was reportedly killed, along with his wife, at the headquarters on Wednesday.
Other officers assigned to the BDR are also reported to have abandoned their posts for fear of attack.
It was not clear why an earlier amnesty deal broke down.
Mohammad Qamrul Islam, the state minister for law and parliamentary affairs, had said the guards were surrendering their arms after he emerged from the headquarters early on Thursday.
"We talked to the BDR troops and they said some 50 officers have been killed," he said, but added that he had not confirmed the deaths.
"We heard that the casualties were kept at a hospital inside the compound."
Hospital doctors confirmed that three bystanders were killed and about 20 people injured in the 20-hour mutiny.
Fifty women and children, who had been stranded at the BDR headquarters because of the fighting, were also freed early on Thursday.
The guards say they are upset with their superiors for not raising longstanding demands for equal pay and working conditions as army soldiers during Hasina's visit the previous day.
Kailash Budhwar, a London-based South Asia analyst, told Al Jazeera that the mutiny was "most unexpected".
"And it happened from a unit that was supposed to be most disciplined ... a paramilitary force who guard the border," Budhwar said.
"This certainly brings to memory the dark days when there was turmoil and anti-social takeover, and there are other undercurrents in Bangladesh who might take advantage of the situation."