"Hedi Dalqam, an al-Qaida activist, was arrested at the start of November," said a security official who described him as a "colleague" of al-Ahdal.
The Interior Ministry announced on Friday Saudi-born al-Ahdal had been caught on Tuesday.
In Washington, US officials said al-Ahdal may be implicated in the bomb attack on the USS Cole three years ago which killed 17 American sailors.
The alleged operative, who reportedly lost a leg fighting in the breakaway Russian republic of Chechnya, had not figured on the FBI's list of wanted terrorists. It is claimed al-Ahdal also fought in Bosnia and Afghanistan.
The ministry said security officials had been watching the 32-year-old for several months before surrounding his home west of the capital Sanaa and forcing him to surrender.
Yemeni authorities believe Ahdal was the second-in-command to Ali Qaid Sunian al-Harithi, alias Abu Ali al-Harithi, who was among the six al-Qaida suspects killed in November 2002 in a missile attack by the CIA in Marib province, east of Sanaa.
Harithi has been described by the United States and Yemen as a top al-Qaida leader and planner of the 12 October 2000 attack on the destroyer USS Cole in the port of Aden.
Yemen, the ancestral homeland of al-Qaida leader Usama bin Ladin, has been under pressure from Washington since the 11 September 2001 attacks on the United States to crack down on presumed al-Qaida members.