Lawyers have spearheaded opposition to Musharraf since the former army chief tried to dismiss Chaudhry last year.
Chaudhry and dozens of other judges were dismissed after Musharraf declared emergency rule in November.
"We are out to save the judiciary. We are out to save the country," Mehmood-ul-Hassan, president of the Karachi Bar Association, told the rally as lawyers chanted "Go Musharraf" in a street in the centre of Karachi.
March 9, 2007: President Musharraf sacked the country's top judge Iftikhar Muhammed Chaudry, for what he called "misuse of authority."
March 12, 2007: A boycott of the courts began, lasting several days. Chaudhry started touring the country, gathering support for what he called his "illegal dismissal".
July 2, 2007: The supreme court turned down Musharraf's decision, describing it as "scandalous" and Iftikhar Muhammad Chaudhry was reinstated.
November 3, 2007: Musharraf declared a state of emergency, putting Chaudhry under under house arrest and firing him along with other 60 judges.
February 2008: The new coalition won the general election and Chaudhry was released.
Kamal Hyder, Al Jazeera's correspondent in Islamabad, said: "It is going to be a real test for the government and the judiciary.
"Because this is a make-it or break-it' for the judiciary.
"This is a very big march they have organised and they have had to organise it because ... there are serious divisions within the ruling coalition as to the fate of the president at the moment."
Dubbed a "long march" even though the lawyers will travel in a motor convoy from Karachi to Multan, where the march to Islamabad will officially begin, it is the first major protest the new government will have to contend with.
Both sides have vowed to keep the peace, with the Pakistan government saying that the lawyers have the right to protest.
The protest could also trigger even deeper splits in the coalition led by the party of Benazir Bhutto, the assassinated former prime minister, which is seen as dragging its feet on the restoration of Chaudhry and other sacked judges.