"Inshallah [God willing] we will be meeting off and on. In principle, we have agreed to stay together."
The PPP has gained the most seats in parliament, ahead of Sharif's party.
"The sooner he [Musharraf] accepts the verdict, the better it is for him," Sharif said.
"I think the nation today has given out its verdict, and that verdict is amply clear and it is from every nook and corner of Pakistan."
Zardari said that after swearing in the government would first ask the UN to investigate the December 27 assassination of Bhutto.
Al Jazeera's Kamal Hyder, in Islamabad, said the government will be formed within the next few days and that the Awami Party, who made surprising gains in the elections, was expected to join the coalition.
"What came out of the meeting was that the doors were open for other parties to come on board as long as they would also accept that there would be a UN enquiry into the assassination of Benazir Bhutto."
'Countdown' to presidential exit
Hyder said that Musharraf was in an "extremely dangerous and precarious position" because the public and Sharif are demanding his resignation.
The opposition claims that they can reach the two-thirds majority in parliament that they would need to impeach Musharraf.
"Some local news networks have already said that the countdown for his exit has begun."
Zardari has said that none of the parties allied to Musharraf will be invited to join the coalition.
Lawyers protested across the country on Thursday to be reinstated, many clashing with police who used tear gas in some cases.
Musharraf purged the judiciary of opposition to his rule last year.
Sharif addressed the lawyers and their supporters outside Chaudhry's home in Islamabad.
Chaudhry is being held under house arrest there.
Sharif told the demonstrators: "It is your duty to adhere to the law and not to abide by the orders of Pervez Musharraf who is illegal and unconstitutional."
If Chaudhry returns to his position he could overturn Musharraf's controversial victory in a presidential election in October and oust him as president.
Musharraf has said that he will not resign.