Police arrested him on an extradition warrant issued by the United States, a spokeswoman said.
"We arrested a 47-year-old man at about three o'clock this morning," she said.
He was taken from his home in west London to a central London police station.
Abu Hamza was expected to appear at the high-security Belmarsh Magistrates' Court in London later on Thursday.
The US holds the cleric responsible for the attack on its destroyer Cole in Yemen in October 2000. Abu Hamza is also wanted by Yemeni authorities who accuse him of sending 10 men, including his son, to Yemen to launch attacks on US interests.
Further allegations include links with Yemeni Islamists, and that he played a role in the kidnapping of westerners in Yemen in 1998.
Abu Hamza denies all the accusations against him.
The Egyptian-born cleric, who was stripped of his British nationality a year ago because of allegations he supported terrorism, won nine more months to appeal against the ruling in April.
Al-Masri leading prayers outside
the closed Finsbury Park Mosque
Islamist Abu Hamza al-Masri became the first person to have his British citizenship revoked after new measures were brought in last year to deport dual-nationality immigrants whose words or deeds are deemed to "seriously prejudice" British interests.
The son of an Egyptian army officer, Abu Hamza al-Masri arrived in Britain in 1979.
He became a British citizen after marrying a British woman in 1981. In 1989 he left for Afghanistan and stayed there for three and a half years.
He returned from Afghanistan with one eye and a steel hook in place of his right hand after wounds from fighting in Afghanistan, injuries that have allowed Britain's tabloid press to constantly parody him.
He then became a preacher in North London's Finsbury Park mosque. The mosque was raided and closed by anti-terrorism police last year. But Abu Hamza continued to preach on the street outside the mosque.