Klein's release comes less than two weeks after a Frenchman jailed with him, Stephane Lherbier, was freed from Tehran's Evin prison after serving most of his 18-month sentence for the same infringement of Iran's territorial waters.
Lherbier was the captain of a boat taking Klein on a trip from the United Arab Emirates across the Gulf.
The two men, who were arrested in November 2005, said they had been misled by Emirati maps showing the waters as Emirati, but were both sentenced to 18 months in jail two months later.
France said Lherbier, who was freed on February 28, had been granted clemency by Iran's supreme leader Ayatollah Ali Khamenei.
After his release, the Frenchman and the German foreign ministry urged Tehran to free Klein as well.
Klein's wife had asked the Iranian authorities to grant her husband clemency, saying he was in fragile health.