Chaudhry and dozens of other judges were dismissed after Musharraf declared emergency rule in November.
"We are out to save the judiciary. We are out to save the country," Mehmood-ul-Hassan, president of the Karachi Bar Association, told the rally as lawyers chanted "Go Musharraf" in a street in the centre of Karachi.
Dubbed a "long march" even though the lawyers will travel in a motor convoy from Karachi to Multan, where the march to Islamabad will officially begin, it is the first major protest the new government will have to contend with.
Both sides have vowed to keep the peace, with the Pakistan government saying that the lawyers have the right to protest, but the possibility of violence cannot be ruled out.
The protest could also trigger even deeper splits in the coalition led by the party of Benazir Bhutto, the assassinated former prime minister, which is seen as dragging its feet on the restoration of Chaudhry and other sacked judges.