Leaders of the main parties in parliament accused Musharraf of dismissing around 60 senior judges, many of whom have been under house arrest since, amid fears that they would overturn his re-election. Misconduct allegations
Musharraf said Chaudhry was sacked over allegations of impropriety and misconduct.
Chaudhry's suspension had sparked the country's biggest political crisis since Musharraf seized power in a bloodless coup in 1999.
Athar Minallah, Chaudhry's aide, said that the sacked judge was grateful to Gilani.
"I have just met the chief justice, he has thanked the prime minister and the new assembly and he has also said that he is praying for the democratic institutions," Minallah told the AFP news agency.
"From tomorrow he will start meeting people and thanked the common people who have come to greet him. He has asked his supporters not to raise any slogans in the judicial colony, to respect the sanctity of the judiciary." Musharraf 'isolated'
Al Jazeera's Kamal Hyder, reporting from Islamabad, said Gilani's appointment is a glimmer of hope for Pakistan.
"This [call for judges release] is a victory for the electoral process, and those who had campaigned for the restoration of the judiciary," he said.
Hyder also said that the newly-elected coalition will isolate Musharraf.
"The president asked the public to vote for him in the elections, and an overwhelming majority did not.
"Gilani will bear many responsibilities, and he will have his work cut out, attempting to restore the relevance and the supremacy of the parliament," he said.
The coalition led that won general elections in February has vowed to reinstate the sacked judiciary.
If the new government honours that pledge, the judges could still Musharraf's re-election illegal and remove him from the presidency.
The new prime minister also said he will ask parliament to pass a resolution seeking a UN inquiry into the assassination of Benazir Bhutto. Overwhelming win
Gilani won 264 votes in the 342-seat lower house of parliament, while Chaudhry Pervaiz Elahi, a senior leader of the pro-Musharraf Pakistan Muslim League (PML-Q) party, secured 42 votes in the poll on Monday evening.
Fahmida Mirza, the parliament speaker, said: "Yousaf Raza Gilani commands the majority of the members. Please come forward and take the seat of leader of the house."
He will be sworn in on Tuesday by Musharraf.
There had been speculation that the PPP would nominate a stop-gap prime minister and Asif Ali Zardari, Bhutto's widower, who now leads the party would take over the post if enters parliament after standing in a by-election.
But the News
, a Pakistani newspaper, on Monday cited Zardari as rejecting such speculation and saying that Gilani would be prime minister for a full five-year term. Time in jail
Gilani was jailed in 2001 by the Musharraf government for making illegal appointments but was freed in 2006. He said the charge was politically motivated.
While in prison, Gilani wrote a book that advocated a strong military, but one that was removed from politics.
He has called for the repeal of constitutional changes made by Musharraf to bolster his authority, including the power to dismiss a government.
For his part, Musharraf has promised full support to the incoming government.
"My brothers and sisters, you are seeing that a real democratic era has begun in Pakistan," he said while addressing the Pakistan Day military parade in Islamabad, the capital, on Sunday.
"Which ever new government is formed, it will enjoy my full support."