Information Minister Sheikh Rashid on Wednesday said Aziz had a large lead in both Attock district in the Punjab province and the desert constituency of Tharparkar in southern Sindh province.
"His victory is absolutely certain," Rashid said.
The by-elections were held to elect Aziz to the lower house so that he can take up the post of prime minister next week.
The final results are expected later in the night.
Aziz will replace Chaudhry Shujaat who took over temporarily after prime minister Zafarullah Jamali resigned on 26 June.
Aziz – a former Citibank executive – survived an assassination attempt on 30 July and has no political affiliation.
The information minister said he expected the new cabinet to be sworn in by 25 August.
Opposition parties are furious that Aziz has been effectively hand-picked by Musharraf, whose power they say is illegal.