Earlier on Wednesday Bangladesh's supreme court rejected the men's appeals against their death sentences.
Six other former officers have been convicted in their absence, while another died abroad.
Hundreds of police and special forces were deployed outside the prison, where a crowd, including Rahman's supporters gathered to chant slogans to celebrate the outcome.
Mujibur, or Mujib as he was popularly known, was killed along with most of his family in a 1975 revolt by a group of young army officers.
Four years earlier Bangladesh had won independence under Mujibur's leadership following a nine-month war in which three million people died, according to official records.
One of Mujib's two surviving daughters, Sheikh Hasina, is now the country's prime minister.
Hasina and her sister, Sheikh Rehana, survived the 1975 killings because they were abroad at the time.
Their mother, three brothers and several relatives were killed.
The men first went on trial after Hasina took office in 1996, but the process came to a halt after Begum Khaleda Zia, her rival, came to power in 2001.
After assuming office for a second term in January last year, Hasina vowed to speedily complete the trial of those accused over her father's death.
The five were eventually convicted but appealed the judgement on the grounds that Mujib's death was part of a mutiny and they should have been tried under martial law instead of through the civilian court system.