Such an outcome would harden a North-South split undermining the next IAEA chief's authority, he added.
El Baradei, an Egyptian diplomat, has been in the job for 12 years and served three terms and will be stepping down as director-general in November.
His successor will assume the highly sensitive nuclear dossiers of Syria, Iran and North Korea and will also have to persuade IAEA member countries to contribute more money to its budget.
The race to replace him narrowed when Jean-Pol Poncelet, a former Belgian deputy prime minister and now an executive at Areva, the French nuclear group, withdrew his name.
Poncelet finished last in a June 9 straw poll among five men, along with Slovenia's Ernest Petric, who on Tuesday dropped out of the race.
Past votes on the IAEA's 35-nation governing board have split along divisions between rich and poor nations.
El Baradei, who won the the Nobel Peace Prize in 2005 for his work at the IAEA, has often been criticised as being too outspoken and has also been accused of politicising the agency.