On Saturday, the federal government imposed a ban on Sunday's Islamabad march after similar protests in Pakistan led to violence in which at least five people have been killed in the past week.
The Muttahida Majlis-i-Amal (MMA), an alliance of six Islamist parties, had said its followers would defy the ban.
Despite police cordons, teargas and warning shots, around 1000 protesters managed to congregate near a central bazaar where they chanted religious and anti-government slogans.
Demonstrators lampooned Pervez Musharraf, Pakistan's president, as a lackey of George Bush, the American president.
"Bush has reared a dog wearing a uniform," they chanted, referring to Musharraf's broken promise to the MMA that he would give up his dual role as president and chief of the armed forces, and to his alliance with Washington in a war on terrorism.
Police and paramilitary troops patrolled streets in Islamabad, while barbed wire was placed across main routes leading to parliament and elsewhere.
Buses and other vehicles were stopped and searched at entry points into the city, witnesses said.
But around 100 protesters broke through the cordons and were joined by hundreds of others from side streets.
Police fired teargas, and when the protesters retaliated by throwing stones, police responded by firing warning shots into the air and what a local official said were rubber bullets into the crowd. Helicopters later flew over the area.
Earlier, on Sunday morning police put Qazi Hussain Ahmed, MMA's president, under house arrest in Lahore before he could travel to Islamabad to lead the march.
Rehman (L) said until there is an
apology, protests will continue
Fazul-ur-Rehman, another senior MMA leader, and a group of around 30 followers, including parliamentarians, assembled at one of the main entry points to Islamabad, but were forced to abandon their march after police fired teargas.
Islamist parties have seized on the issue because they say Islam forbids images of Mohammad.
Rehman said: "The whole Pakistani nation will continue and the Muslim world will continue coming on (to) the streets until an apology is offered for this sinister act and a promise made not to repeat it."
MMA supporters burned tyres on roads in the nearby city of Rawalpindi, where police had earlier detained more than 100 activists.
Pakistan has issued diplomatic protests over the cartoons published in several, mainly European newspapers.
On Friday, Pakistan recalled its ambassador from Denmark, where the cartoons first appeared.
The Danish Foreign Ministry said on Sunday that its ambassador had come home temporarily, having two days earlier shut the embassy in Islamabad because of the security risk.
"Our rulers should immediately recall ambassadors from countries where these cartoons have been published"
Abdul Waheed, a protester, said: "Our rulers should immediately recall ambassadors from countries where these cartoons have been published.
"We don't ask for severance of diplomatic relations but only recall of ambassadors to put pressure on those countries."
The editor of the Danish paper that started the controversy, Jyllands-Posten has apologised, and the apology was printed in Saudi Arabian newspapers on Sunday.
A leading cleric in the northwestern Pakistani city of Peshawar has offered a reward to anyone who kills a Danish cartoonist responsible for the insult.