The remains in Aden show that some of the men had their hands bound before being killed execution-style. They could date back 20 years to the country's civil war.
There has been no official reaction yet to the find.
The remains of the men, in military uniform and covered in blankets, were found during a dig at a plot of land near Al-Sulban barracks in Al-Areesh suburb, northeast of the port city.
The remains were taken to an Aden hospital for tests. An official at the hospital where the bodies were taken said 16 bodies were unearthed late on Saturday and another eight on Sunday.
Captain Ahmed al-Maarabi of the local police said: "Their ages could be between 25 and 35 and from their uniforms they could have belonged to the mukhabarat (secret police) or the air force police."
The men could have been victims
of the country's civil war
Abdullah Qairan, the head of public security in Aden, said the men could have been killed during a January 1986 coup in the former South Yemen that overthrew its then leader Ali Nasser Mohammed, who is now living in Damascus.
South and North Yemen united in 1990 under the leadership of Ali Abdullah Saleh, the then president of the north. The divided countries were also immersed in a border conflict in 1979 and 1972 that left hundreds dead. They likewise fought a north-south civil war in 1994.