The tape, posted on Thursday in a website often used by Iraqi insurgent groups, could not be authenticated.
"O Sunnis! Prepare to get rid of the infidel snakes and their poison ... and don't listen to those advocating an end to sectarianism and calling for national unity. This is a weapon to get you to surrender," said the speaker on the tape who sounded like al-Zarqawi.
Iraq's new national unity government vowed last month to rein in violence and heal the country's sectarian wounds.
The speaker blasted Iraq's top Shia cleric, Ayatollah Ali al-Sistani, as the "leader of infidelity and atheism" and said his followers were more concerned about honouring their own saints than protesting against cartoons of Prophet Muhammad published in Danish and other newspapers around the world.
"We did not see them rise up with the same fervour when blasphemous pictures of the Prophet were published because they prefer their own leaders to God and his Prophet," he said.
The speaker accused Shia groups and government forces of being responsible for numerous attacks on Sunnis and their places of worship.
He suggested that Shia themselves were behind the February bombing of a Shia shrine and other attacks which touched off a wave of sectarian killings and revenge attacks.
"The attacks were a charade ... that revealed their (Shia) hatred of the Sunnis," the speaker said.
The tape, which was issued in three parts totalling about four hours, covered what the speaker said were examples of Shia enmity towards Islam throughout history.
The speaker criticised a militia loyal to Shia cleric Muqtada al-Sadr for stopping their fight against US forces after uprisings in 2004 against US troops.
The speaker also attacked Lebanon's Shia group Hizbollah, and said majority-Shia Iran had helped the United States in Afghanistan and was in contact with Washington over Iraq.