The men were sentenced last year on various charges of terrorism, an official said, speaking on condition of anonymity because he was not authorised to speak to the press.
They were being held at a detention centre for military intelligence in Sanaa.
No further details were available.
The escape on Friday came a day before the trial of Muhammad Hamdi al-Ahdal, an al-Qaida suspect, and 14 others charged with involvement in operations in Yemen, particularly the 2000 attack on the USS Cole.
Al-Ahdal is suspected of planning the Cole bombing, which killed 17 US sailors, and the bombing of a French oil tanker off Yemen's coast in 2002, which killed a Bulgarian crew member and spilled 90,000 barrels of oil into the Gulf of Adan.