The five were arrested on Tuesday, the security official in Abyan province said.
Two were fugitives who escaped the sweep in the Jabal Hatat region last month, and the three others were arrested in Abyan for sheltering some of those wanted, he said.
Yemen launched the major operation in Jabal Hatat on 24 June, deploying hundreds of soldiers, tanks, artillery, heavy machine-guns and helicopters in the region 120 km northwest of the southern city of Aden.
Six suspects and one police officer were killed in the clashes and the interior ministry reported seizing a large quantity of arms. The troops arrested 21 fighters.
The leader of the fugitives, Khaled Abdennabi, was among those killed, according to officials.
After mediation failed to secure a surrender, Yemeni special units went into action against the group accused of having carried out an attack on an army medical convoy that left seven wounded on 21 June.
The dozens of suspected fighters hiding out in the rugged and largely inaccessible region included members from the Islamic Jihad group and the Islamic Army of Aden-Abyan, as well as sympathizers of Usama bin Ladin's al-Qaida network, according to military officials.
Yemen is the ancestral homeland of bin Ladin. Its government has been under pressure from Washington since the September 11 attacks on the United States to crack down on suspected al-Qaida members, and has arrested dozens of them.