An official said the alleged smugglers included three Australians, one Danish national, a Briton and a German, but gave no details on the identities of the two others.
"The eight foreigners were arrested because they smuggled weapons to Somalia from Yemen," the official said in a statement posted on the state-run news agency Saba.
"Preliminary investigations indicate that they are members of al-Qaeda."
Government sources in the Yemeni capital, Sanaa, told Reuters news agency that the eight men had converted to Islam earlier this year and received religious instruction in the country.
A security official told Associated Press that at least four of the men had been studying at the Iman University, which is run by Sheikh Abd al-Majid al-Zindani. The US considers al-Zindani to be an al-Qaeda supporter.
A Danish foreign ministry official confirmed the arrest of the Dane, but refused to identify him.
"All I can say is that a Danish national has been arrested according to our information," said Uffe Wolffheckel of the foreign ministry's consular service.
Danish media said the suspect is a 23-year-old male who converted to Islam and moved to Yemen two months ago with his wife and child.
Yemen is believed to be a frequent route for smuggling arms to Somali factions.
Last month, Yemen claimed that it had broken up an al-Qaeda-linked cell that was behind foiled attacks on oil and gas installations.