An Aljazeera correspondent at the scene said the man was walking with other journalists at the march when an assailant walked up behind him and shot him once in the back with a handgun, before disappearing into the crowd.
The Swedish foreign ministry identified the photojournalist as Martin Adler.
Adler worked for, among others, Swedish tabloid Aftonbladet, both as a writer and a photographer, the paper confirmed.
Adler was taken by other journalists and bystanders to a nearby hospital but died on the way, Aljazeera's correspondent added.
Television footage showed Adler on the ground with a gunshot wound to his back, surrounded by shocked bystanders.
Adler was covering a demonstration organised by the Islamic Courts Union, which wrested control of the city from US-backed regional commanders this month following weeks of heavy fighting in which at least 360 people died.
Abdirahim Isse, an aide to the courts group leadership, said an unarmed woman who was close to the victim and ran away had been arrested for questioning.
Speaking to Aljazeera, Sheikh Sharif Sheikh Ahmed, chairman of the courts union, condemned the killing and accused "certain parties" of seeking to foment tension to pave the way for the deployment of foreign troops in Somalia.
The killing is the first of a foreign journalist in the city since BBC producer Kate Peyton was shot dead outside a hotel in Mogadishu in February 2005.
A speaker at the rally on Friday said those responsible would be punished.
Heavy fighting in recent months
have left 360 people dead
"We are against the killing of a journalist who is a guest," he told the 4,000 strong crowd.
"This person deserves to be punished for killing somebody for no reason."
Anti-foreigner sentiment is high in the country after reports that regional commanders were financed by the CIA to capture suspected al-Qaeda members in Somalia.
Foreign journalists have been stoned or heckled while reporting on demonstrations recently.
Since the courts took over, several Western journalists have gone into the city - previously considered too dangerous to visit - at the invitation of the courts group, which says it is bringing peace and order to a country in desperate need of calm.
On Thursday, Somalia's transitional government reached an agreement with the courts group, with both sides agreeing to recognise each other and to lay down arms.
The rally was staged in support of the declaration of principles signed in Khartoum between the courts group and the transitional government, under the sponsorship of the Arab League.