Two Bangladeshi nationals identified as Ahmad Faisal, 26, and Muhammad Mukhtar Husain, 29, along with Kane Yahya, a 41 year old man from Mali, had been staying illegally in Japan, the Tokyo District Prosecutors Office said in a statement.
Faisal sneaked into the country after travelling by ship from South Korea around January 1997, and Husain came to Japan in February 2000 on a fake passport, it said.
Yahya overstayed his visa, which expired in October 1989, the statement said, without elaborating.
Officials at the office declined to comment further, only saying the three foreign nationals were believed to have links with al-Qaida.
The three were among eight foreigners arrested in Japan in connection with an investigation into Lionel Dumont, 33, a Frenchman linked to al-Qaida who stayed in Japan after the 11 September attacks in the United States, news reports said.
Dumont allegedly belonged to Al-Qaida's logistics arm and police suspect he was engaged in raising funds, money laundering and forming a ''terrorist network while hiding in Japan'' between July 2002 and September 2003, according to earlier reports.
Police and security agencies have refused to comment on Dumont's reported stay in Japan, although it has been tacitly acknowledged in comments by politicians including Prime Minister Junichiro Koizumi.
Dumont is included on a US Treasury Department list published in June 2003 specifying 17 individuals linked to al-Qaida whose assets had been frozen.
Officials at the office declined to comment further, only saying the three foreign nationals were believed to have links with al-Qaida
He was arrested in Germany in December 2003 for ''non-terrorist'' offences and was extradited earlier this year to France, where, in 2001, he was sentenced in absentia to life in prison for armed robbery.
After dropping out of university, Dumont converted to Islam in 1993 and fought for the Mujahidin alongside the Bosnian army in Bosnia-Hercegovina.
He returned to France and is suspected of forming the "Roubaix Gang" in 1995, which carried out armed robberies to raise funds for Islamist causes until he fled a police raid a year later.
He reportedly entered Japan in July 2002 despite the country's stepped-up immigration measures after the September 11 attacks.