Muhammad al-Baradai said on Sunday that such a move would reduce frustration in the region caused by "what is seen to be a widespread imbalance".
The International Atomic Energy Agency (IAEA) chief is scheduled to travel to Israel in July to discuss making the Middle East a nuclear-free zone, despite Tel Aviv's refusal to admit it has such WMDs.
"I think everybody takes it as a given that Israel has a nuclear capability, if not nuclear weapons," he said.
"We need ... to rid the Middle East of all weapons of mass destruction," he told reporters on a visit to Russia. "Israel agrees with that, but they say it has to be ... after peace agreements.
"My proposal is maybe we need to start to have a parallel dialogue on security at the same time when we're working on the peace process."
Al-Baradai added he would like Israel to open up nuclear facilities to inspections by the IAEA.
International analysts believe Tel Aviv has built more than 100 nuclear weapons.
Its Arab neighbours have frequently accused the international community of double standards for requiring them to be free of nuclear weapons while doing little about Israel.
Al-Baradai concluded it was "not sustainable in any region or even globally to have some [people] rely on nuclear weapons and others being told they should not have nuclear weapons".