Justice Javed Iqbal, the next most senior judge of the Supreme Court, was appointed as acting chief justice, state-run Associated Press of Pakistan news agency said.
The president has submitted a case against Chaudhry to the Supreme Judicial Council, APP reported.
The council is a panel of top Pakistani judges that adjudicates cases brought against serving judges and will decide whether the charges against Chaudhry merit his formal dismissal and whether he should be prosecuted.
Musharraf had received "numerous complaints and serious allegations for misconduct, misuse of authority and actions prejudicial to the dignity of office of the chief justice of Pakistan", and Chaudhry had been unable to give a satisfactory explanation, APP said.
The report did not specify what he was accused of and Chaudhry was not immediately available for comment.
On Saturday, scores of Pakistani lawyers demonstrated in protest of the decision, boycotting court sessions in Quetta, Karachi and Lahore and burning effigies of Musharraf.
Recent rulings by Chaudhry may have irked the government.
Last June, the Supreme Court rejected a government move to sell 75 per cent of state-owned Pakistan Steel Mills to a Saudi-Russian-Pakistani consortium for $362 million. Mill workers claimed it was greatly undervalued.
Also, Chaudhry has heard a landmark case brought by relatives of dozens of people believed taken into secret custody by Pakistani intelligence agencies. The chief justice has pressed the government to provide information on the detainees whereabouts.