The group's $300,000 share of the ransom was also lost, he said.
Three pirates reached shore after swimming for several hours, pirate Daud Nure said on Saturday.
"There has been human and monetary loss but what makes us feel sad is that we don't still have the dead bodies of our relatives. Four are still missing and one washed up on the shore," Abukar Haji, the uncle of one of the dead men, said.
The Sirius Star, owned by the shipping arm of oil giant Aramco, was carrying crude oil valued at $100m when it was hijacked on November 15.
Saudi Arabian oil minister Ali Naimi said on Saturday that the crew was safe and returning to the Gulf coast city of Dammam.
An Iranian-chartered cargo ship was also released by pirates, Iran's state television reported.
It said the ship Delight was carrying 36 tons of wheat when it was attacked in the Gulf of Aden on November 18 and seized by pirates.
All 25 crew are in good health and the vessel is sailing toward Iran, the TV report said.
The US Navy announced this week that it will head a new anti-piracy task force after more than 100 ships were attacked last year.
Other countries, including Russia and India, have sent naval ships to patrol waters around the Horn of Africa as acts of piracy increasingly threaten the safe passage of maritime cargo through the Gulf Aden towards the Suez Canal.
Somali pirates are holding 17 ships and more than 250 crew members hostage, and have carried out more than 100 attacks in the last year.