He described Musharraf's rule as illegitimate and said he would go straight to Lahore, his base of power in the east of the country, after landing in Islamabad.
Benazir Bhutto, another former prime minister, said on Wednesday that she had almost sealed a power-sharing deal with Musharraf under which he will quit as Pakistan's army chief.
"Musharraf is shouting slogans of war against terror in order to win US support"
Ali Abu Hamza, Chichawatni, Pakistan
Send us your views
But a presidential spokesman said on Thursday that Musharraf had yet to decide whether to step down as army chief.
Pakistan's Supreme Court ruled last week that Sharif, a two-time prime minister, should be allowed to return.
Sharif has vowed to oppose a bid by Musharraf for another term as president.
After he was overthrown, Sharif was sentenced to life in prison on graft and security charges. He was exiled in 2000.
With his term coming to an end and his popularity plummeting, Musharraf has turned to Bhutto for help to shore up his position, and his aides are meeting with the two-time prime minister in London.
But Bhutto has insisted an agreement would hinge on Musharraf stepping down as chief of the army, which has ruled for more than half Pakistan's history since independence in 1947.
In an interview with the Financial Times published on Wednesday, Sharif, who is leader of the Pakistan Muslim League, called Bhutto's attempts to deal with Musharraf a "setback" and a "clear violation" of a deal agreed between them to do "no deals with military dictators".
In another development with potential political ramifications, Pakistan's supreme court agreed on Wednesday to hear an appeal against Musharraf's role as army commander.
The application was filed by Qazi Husssain Ahmed, the chief of Pakistan's main coalition of Islamist parties - the Muttahida Majlis-e-Amal, or United Action Front.
|Bhutto and Sharif had met in 2006 and agreed|
to oppose Musharraf's rule [GALLO/GETTY]
Javed Iqbal, a judge of the court, said: "The petition is fixed for regular hearing."
He said that objections from the court registrar's office that Ahmed had no authority to lodge the appeal, were "devoid of merit".
Musharraf, who seized power in a bloodless military coup in 1999, is seeking re-election as president-in-uniform by parliament in September or October.
"The people are fed up with the unconstitutional and dictatorial military regime," Ahmed said outside the court.
The application says that under military regulations, Musharraf's term as chief of army staff had expired in 2001.
The application also states that he was no longer eligible to continue in the post after turning 60 in August 2003.
The legal challenge argues that in 2004, Musharraf had broken a public pledge to hang up his military uniform.
Musharraf has suffered a series of legal setbacks since his attempt in March to sack Iftikhar Muhammad Chaudhry, the chief justice of the supreme court.
The court had reinstated Chaudhry in July after months of nationwide protests and then ruled last week that Nawaz Sharif, a former prime minister, could return from exile.