"You must ambush, mine, raid and [carry out] martyrdom campaigns so that you can wipe them out," he said.
Ethiopian forces helped Somalia's under-powered forces loyal to the interim government to rout the courts in a two-week war.
The tape was posted on a website used by armed Islamist groups. Its authenticity could not be verified, but it is believed that the voice was that of al-Zawahri.
The Union of Islamic Courts, which took control of the capital in June and much of the south of the country, abandoned Mogadishu last week in the face of advancing Ethiopian and government forces.
"You must ambush, mine, raid and [carry out] martyrdom campaigns so that you can wipe them out"
al-Qaeda deputy leader
The courts said its retreat was tactical and designed to avoid major bloodshed.
Since its fighters left, one Ethiopian soldier has been killed in an ambush in the south and a hand grenade was thrown at Ethiopian troops in the capital on Thursday.
The US and Ethiopia have portrayed the Islamic courts as linked to, and even run by, al-Qaeda, putting Somalia firmly on the map of the US-led "war on terror".
Kenya closes borders
Kenya has already closed its border with Somalia saying it wants to stop fighters slipping across the frontier. The UN says thousands of refugees, also near the border, have been prevented from getting into Kenya to avoid the fighting.
Residents of Mogadishu have been on edge since the government took over with the crucial military aid of neighbouring Ethiopia. Some of the feared gunmen of the past have returned to the city.
Al-Zawahri criticised the UN Security Council's failure to issue a resolution demanding that Ethiopian troops withdraw from Somalia and said plans to send peacekeepers to the Horn of Africa provided political cover for Ethiopia's invasion.
Jendayi Frazer, assistant US secretary of state for Africa, said on Thursday that Yoweri Museveni, the Ugandan president, had promised George Bush in a recent telephone call that he could supply between 1,000 and 2,000 troops to protect Somalia's transitional government and train its troops.
The interim government wants a foreign peacekeeping force, approved by the UN before the war, to be deployed as soon as possible.
Ban Ki-moon, the newly appointed UN secretary general, called on Thursday for its quick deployment and welcomed Ethiopia's plan to pull its troops out of Somalia in a few weeks.
Al-Zawahri urged Islamist groups in Yemen, Gulf Arab states, Egypt, Sudan and north Africa to help those fighting in Somalia.
"I call on my Muslim brothers everywhere to fulfil the call for jihad in Somalia," he said.