A police source said on Wednesday the 29-year-old was arrested at Munich station as he was about to leave the city.
The public prosecutor's office in Munich said the man, identified only as Muhammad L, was under an arrest warrant for having contravened immigration laws by bringing two compatriots into Germany.
The Sueddeutsche Zeitung said the man was suspected of having organised the travel of a dozen people to Iraq to commit "suicide attacks" against US forces.
The public prosecutor refused to comment on that allegation, saying that investigations were only just underway.
Sources from German internal intelligence services told the newspaper about 100 Ansar al-Islam militants were present in Germany, mainly in the south of the country.
Cells existed in Berlin, Cologne, Duisburg, Frankfurt, Hamburg and large towns in Bavaria, the paper said.
Another suspected member of the Bavarian cell, named "Doctor Omet", has been in custody since March, it added.
US officials contend Ansar al-Islam has loose links to al-Qaida, the network they accuse of being responsible for the September 11 attacks.
The group's stronghold in northern Iraq - a cluster of 16 villages - was devastated by US air strikes in early April during the US invasion of Iraq.
But US commanders have said Ansar al-Islam has made a strong comeback, infiltrating Iraq from Iran and setting up operations in the Baghdad area.