In the meantime, the panel of five judges adjourned its hearing until April 24.
Besides demanding a public trial, Chaudhry's lawyers have challenged the impartiality of three of the five panel judges.
The judicial crisis is the biggest challenge to the authority of Musharraf since he came to power in a military coup seven years ago.
"It would be in the best interests of Pakistan for Musharraf to step down"
Jim ibarra, Cyberjaya, Malaysia
Send us your views
Opposition groups and the legal community see the government's attempt to sack Chaudhry as an attack on the judiciary's independence.
Wednesday's rally saw nearly 1,500 opposition activists and lawyers on the street outside the supreme court.
Police officers and paramilitary troops with batons, helmets and riot shields stood at the road side outside the court, but did not intervene.
"We are here one, to strengthen the resolve of the chief justice... and two, to send a message to the other judges, look, the public will not tolerate judgement on the directives of GHQ [army headquarters]," said Imran Khan, leader of the Tehrik-e-Insaf (Movement for Justice) party.
The government has not specified the accusations against Chaudhry but a newspaper has reported the main one appeared to be he used his position to help his son get a public sector job.
Analysts say Musharraf's move to sack Chaudhry could have been motivated by fears he would not allow the president to renege on a commitment to step down as army chief this year, as is written in the constitution.
Musharraf plans to seek another term as president amid speculation he is reaching out to Benazir Bhutto, a self-exiled former prime minister, to form a common front against conservative religious forces.