"There was a major breakthrough in the talks late last night. We have agreed to impeach the president," said a senior member of Nawaz Sharif's Pakistan Muslim League-Nawaz (PML-N) party.
The coalition has been split by the issues of what to do about Musharraf and how to carry out their promise to restore senior judges dismissed by Musharraf under emergency rule.
An official from the late former prime minister Benazir Bhutto's Pakistan People's Party, which is now led by her widower Asif Ali Zardari, confirmed the decision.
Officials said that the decision came when Sharif assured Zardari that he could count on the support of some former members of the PML-N who are currently members of a pro-Musharraf party.
The decision was quickly followed by the announcement of the cancellation of the president's trip to the Beijing Olympics - for the second time.
Yousuf Raza Gilani, the prime minister, will go instead, the foreign ministry said on Thursday.
"Yes, the president has cancelled his visit to China and the prime minister is now going," Mohammad Sadiq, the foreign ministry spokesman, said.
Musharraf was initially scheduled to leave on Wednesday for the Olympic opening ceremony but cancelled his flight.
The foreign office had earlier announced that he was due to leave on Thursday.
Local media had reported that during the first day of talks on Tuesday, they had agreed to ask Musharraf to resign and then impeach him through parliament if he refused.
Reports also said they were spurred into action by fears that Musharraf may act first and dissolve parliament.
Ameen Jan, an analyst on Pakistan, told Al Jazeera that Musharraf may try measures that are constitutionally available to him.
"He has the powers under the constitution to dismiss the parliament, which means that the government would also stand dismissed," he said.
"Politically it would be a bold move on Musharraf's part given the fact that his credibility in the country is very low."
But Chaudhry Pervez Elahi, a close ally of Musharraf, said that there was no concern in the presidential camp.
"We will fully defend any impeachment motion," he said.
"There is no panic and everything is normal in our camp."
Musharraf has resisted growing pressure to quit in recent weeks, saying he was willing to work with the coalition to tackle problems such as the ongoing conflict in the country's tribal areas, as well as soaring food and fuel prices.