The federal prosecutor's office identified the arrested men as Ibrahim Muhammad, a 29-year-old Iraqi, and Yasir Abu, a 31-year-old stateless Palestinian.
Police raided four homes in the western cities of Mainz and Bonn early on Sunday. The men, both arrested in Mainz, are accused of belonging to a foreign terrorist organisation. Both had been under surveillance since October.
Federal prosecutor Kay Nehm said the Iraqi had recruited the second suspect in September 2004 as a future bomber in Iraq. He said there was no indication they planned an attack in Germany.
They planned to pretend that Yasir Abu had been killed in a car accident in Egypt and claim more than 800,000 euros ($1 million) from a life insurance policy to fund the bombing and other attacks.
The prosecutor said the Iraqi had also sought to acquire nuclear material for use in an attack.
He said the suspect had been in contact with a group in Luxembourg which had nuclear material, but added that it was not suitable for a bomb.
Both men were due to be brought before an investigating judge on Monday.