Hakim Al-Masmari, editor-in-chief of the Yemen Post in the capital, Sanaa, said at least 12 people were killed and six injured in the clash.
He said the attack targeted the biggest intelligence headquarters in Aden.
"The [assailants] attacked because they believe those who are imprisoned there have been imprisoned against the law," he told Al Jazeera.
"These rebels, according to some human rights activists, are doing what they think is right, because they believe that those who are imprisoned in the facility are imprisoned illegally.
"Some of them have been there for years now without any court hearing or trial."
Following the attack, unofficial sources in Yemen told Al Jazeera that Ali Abdullah Saleh, the country's president, fired two senior security officials for security lapses in the wake of the incident.
There was no immediate claim of responsibility for the attack, but Al-Masmari told Al Jazeera that the attackers could be from an anti-government movement based in the south of Yemen, or an al-Qaeda offshoot.
The attack came a day after Al-Qaeda in the Arabian Peninsula (AQAP), an al-Qaeda affiliate, threatened to respond to a government crackdown against its fighters in eastern Yemen.
It said on Friday that it would "light up the ground with fire under the tyrants of infidelity in the regime" of Ali Abdullah al-Saleh, Yemen's president.
The threat came in response to what the group described as aggression against women and children in Wadi Obeida in the province of Maarib, east of Sanaa.
"We ... will not stand idle over what is happening to our women and children in Wadi Obeida," the US monitoring group SITE quoted a statement from the group as saying.
In late May, Jaber al-Shabwani, a provincial official and three of his bodyguards were killed in a botched air strike in Maarib province that reportedly targeted a wanted member of the al-Qaeda offshoot.
The target of the attack, Mohamed Said bin Jardan, was reportedly wounded but managed to escape, according to Yemeni security services.
Members of al-Shabwani's tribe responded to his killing by attacking the pipeline that carries oil from Maarib to Ras Isa, a terminal on the Red Sea coast.
Yemeni tribesmen attacked an oil pipeline, petrol stations and government installations after the raid to avenge the death of al-Shabwani.
AQAP said women and children had died in the air raids, which had also destroyed homes.