In a taped message broadcast on Aljazeera, Ayman al-Zawahiri said al-Qaeda would not stand by while "these [Israeli] shells burn our brothers" in Lebanon and Gaza.
He called on Muslims to join forces and fight what he called the "Zionist-crusader war" against Muslim nations.
"Oh Muslims everywhere, I call on you to fight and become martyrs in the war against the Zionists and the crusaders," the Egyptian born former doctor said.
"The war with Israel does not depend on ceasefires... It is a jihad for God's sake and will last until religion prevails ... from Spain to Iraq," al-Zawahiri said.
"The entire world is an open battlefield for us and since they are attacking us everywhere, we will attack everywhere."
The deputy to Osama bin Laden wore a grey robe and white turban during the statement. A picture of the World Trade Centre on fire was on the wall behind him along with pictures of two fighters.
"The shells and rockets ripping apart Muslim bodies in Gaza and Lebanon are not only Israeli, but are supplied by all the countries of the crusader coalition. Therefore, every participant in the crime will pay the price," al-Zawahiri said.
He also suggested that the world cared more about Israelis than Palestinians.
"The 10,000 Palestinian prisoners in Israel's prisons do not move anything while three Israeli prisoners have shaken the world," he said.
The statement was the first from al-Qaeda to comment on Israel's offensive in Lebanon which began after the capture of two Israeli soldiers in a cross-border raid by Hezbollah.
Israel also attacked Gaza after Palestinian fighters killed two Israeli soldiers and captured another on June 25.
Al-Zawahiri has evaded capture since US-led forces brought down the Taliban government in Afghanistan in 2001 following the September 2001 attacks on the US.
The message was the tenth released by Zawahiri this year.