Colonel Ubiratan Guimaraes, 63, was found dead in his Sao Paulo apartment. He had apparently bled to death after being shot in the abdomen, police said.
Authorities told Reuters news agency his death did not appear be the result of a failed robbery as his apartment door was unlocked and nothing was stolen.
Guimaraes was sentenced in 2001 to 632 years in prison for sending in a troop of riot police to quell a 1992 uprising in Brazil's infamous Carandiru prison.
However, his conviction was overturned by a Brazilian appeals court in February on a technicality, sparking anger amongst human rights activists and victims' families.
The case became a symbol of police brutality in Brazil, where extra-judicial killings of suspected criminals are commonplace.
According to court testimony about the Carandiru massacre, police slaughtered unarmed prisoners within the prison shooting them at close range even after they surrendered.
Guimaraes said during his trial that his troops were forced to open fire to defend themselves against rioting prisoners, some of whom were carrying guns.
Guimaraes' death comes less than a year after the slaying of the warden of Carandiru prison.
The notorious First Capital Command prison gang was blamed for Jose Ismael Pedrosa's death, according local media reports.
Brazilian authorities did not say whether they believed Guimaraes' death might be linked to the Sao Paulo prison gang, accused of orchestrating three waves of violence in Brazil's biggest city this year that left more than 200 people dead.
Carandiru, once Latin America's largest prison, was demolished in 2002. A year later the massacre was portrayed in an acclaimed film "Carandiru," directed by Hector Babenco.
Brazil's prisons are overcrowded, with bloody riots over living conditions and battles between rival street gangs common occurrences.