Nawabzada Nasr Allah Khan, 85, died of a heart attack on Saturday, his doctors said.
Khan, who headed the Pakistan Democratic Party, brought together the rival parties of former prime ministers Benazir Bhutto and Nawaz Sharif on one platform against the Musharraf’s military government.
His death has raised concerns over the future of the alliance.
Bhutto and Sharif, both living in exile, accepted Khan as the chief of the Alliance for Restoration of Democracy (ARD), after Musharraf led a coup against Sharif in 1999.
Setback to alliance
"It is a serious setback to the alliance," said Farhat Allah Babur, member of the upper house and a spokesman for the Bhutto's Pakistan's Peoples Party (PPP).
"He had kept the two parties with opposing political views together under the ARD," Babur added. "It was his personality that had kept the two rival parties, together under the ARD."
"It was his personality that had kept the two rival parties, with opposing political views, together under the ARD"
Farhat Allah Babur,
The ARD, along with the Muttahida Majlis-e-Amal - an alliance of religious parties - regularly protests and boycotts parliament, demanding that Musharraf, who is president and chief of the armed forces, should quit one of the two posts.
Political analysts say the PPP and Sharif's faction of Pakistan Muslim League (PML-N) are likely to find it difficult to replace Khan with someone acceptable to both.
Chaudhry Shujaat Husayn, president of the ruling Pakistan Muslim League party, said Khan's death had created a vacuum in Pakistani politics.
"He was the last politician of the kind who never bowed before dictatorship. I cannot see any other politician who can take his place," Husayn said.
Khan spent most of his political life in the opposition, cobbling alliances against the government of the day.
He was criticised by his opponents for his party becoming a part of the cabinet of former military ruler General Mohammad Zia al-Haq.
But Khan was later instrumental in forming a democracy movement that campaigned for years against Zia's martial law.