Dragan Nikolic, commander of the Susica camp in northeastern Bosnia in 1992, also admitted facilitating the rape and sexual abuse of several Muslim and other non-Serb women detained at the camp.
The prosecution had asked the International Criminal Tribunal for the former Yugoslavia to jail Nikolic for 15 years in exchange for a belated plea of guilty to charges of crimes against humanity.
But Judge Wolfgang Schomburg told him: "The brutality, the number of crimes committed and the underlying intention to humiliate and degrade would render a sentence such as that which was recommended unjust."
Nikolic, 46, was the first person indicted by The Hague court in 1994, but he was not arrested by NATO-led troops in Bosnia until April 2000.
He was transferred to the court in The Hague the following day and initially pleaded not guilty to all charges.
However, in September this year he changed his plea to guilty of four general charges of crimes against humanity including nine specific counts of murder and five of torture.
Around 250,000 Bosnian citizens died during the 1992-5 Bosnian war, which saw some of the worst scenes of brutality in Europe since World War II.