Shaikh Taj Din al-Hilali, the spiritual leader of Australia's 280,000 Muslims, made the comments during a sermon to mark the end of Ramadan at a mosque in the Sydney suburb of Lakemba.
"Australia is our compassionate mother, and I say to every person living in Australia, from the person in the highest office down to the ordinary man in the street: love this country or leave it, shape up or ship out," he told around 30,000 Muslims gathered at the mosque on Tuesday.
Al-Hilali also condemned the spate of attacks worldwide during Ramadan.
Tension has been rising in Sydney between Muslims and non-Muslims over a crime wave which police blame on young Middle Eastern men, particularly Lebanese youths.
Several murders and revenge killings have been blamed on inter-clan rivalries among the Lebanese community, forcing police to set up a special unit to look into the crimes.
Al-Hilali's sermon followed similar comments by New South Wales Premier Bob Carr last month in which he warned gang leaders to obey the laws of the country or leave it.
Carr, who was at al-Hilali's side at Lakemba on Tuesday, said mutual respect was the key to avoiding a clash of civilisations and that a strong multicultural Australia needed to respect different cultures.
"We want a dialogue with civilisations, we don't want a clash of civilisations, we want the people of the world respecting one another," he said.
The Lakemba mosque has been a focal point of police investigations into the actions of Frenchman Willie Brigitte, who was arrested in Sydney and deported last month under suspicion of plotting attacks.