He also reported that the army has deployed heavy weapons pointing at the guards' compound in Dhaka.
Violence first broke out at the forces' Dhaka headquarters on Wednesday over a pay dispute and other issues.
At least 50 people were reported killed in nearly daylong fighting.
Mohammad Qamrul Islam, the state minister for law and parliamentary affairs, said the guards there had started to surrender their arms after he emerged from the headquarters early on Thursday.
"We talked to the BDR troops and they said some 50 officers have been killed," he said, but added that he had not confirmed the deaths.
"We heard that the casualties were kept at a hospital inside the compound."
Hospital doctors confirmed that three bystanders were killed and about 20 people injured in the 20-hour mutiny.
Fifty women and children, who had been stranded at the BDR headquarters because of the fighting, were freed when the guards began surrendering arms.
The mutineers agreed to lay down arms after Sheikh Hasina Wajed, the prime minister, met a group of them at her residence on Wednesday and offered them amnesty.
She also pledged to look into their demands for better pay and was set to address the nation on Thursday over the issues.
However, negotiations were still continuing on the ground, our correspondent said.
Wednesday's unrest saw the guards seize their headquarters and a nearby shopping mall, and trap dozens of children in a school.
The army was called in and shut down nearby streets as helicopters kept watch over the area.
Intermittent gunfire rang out for more than four hours, while black smoke billowed from the BDR compound.
The guards said they were upset with their superiors for not raising longstanding demands for equal pay and working conditions as army soldiers during Hasina's visit the previous day.
Kailash Budhwar, a London-based South Asia analyst, told Al Jazeera that the mutiny was "most unexpected".
"And it happened from a unit that was supposed to be most disciplined ... a paramilitary force who guard the border," Budhwar said.
"This certainly brings to memory the dark days when there was turmoil and anti-social takeover, and there are other undercurrents in Bangladesh who might take advantage of the situation."