Banners and posters in the party's green colours have been unfurled around the city and Sharif's suburban residence washed and renovated.
Kamal Hyder, Al Jazeera's correspondent in Pakistan, reported that the authorities had already cracked down on Sharif's supporters.
"Last night we are told the government swooped down on supporters of Nawaz Sharif - dozens have been arrested," he said.
A bullet-proof Mercedes car, a gift from king Abdullah of Saudi Arabia where Sharif has been living for the past seven years, was flown in late on Saturday ahead of Sharif's arrival.
In contrast to the last time Sharif tried to return in September, when he was swiftly sent back into exile, Musharraf has given his reluctant approval, according to a senior aide to the president.
Sharif's return will mean further confrontation for Musharraf, who imposed a state of emergency three weeks ago, sending the nation into further political turmoil with opposition leaders split over whether to boycott upcoming general elections.
Sharif returns home, a day after two suicide bombings hit Rawalpindi, a garrison town next to the capital Islamabad, killing up to 35 people.
In the first attack, an explosives-laden car rammed a bus carrying employees of Pakistan's Inter-Services Intelligence agency, then moments later, a bomber hit an army checkpoint in another part of the city, said Mohammed Afzal, a police official.
Intelligence officials were reported as saying at least 35 people were killed, but an army statement said it could only confirm that 15 were killed in the attack on the bus.
It said two security forces personnel were critically injured in the second attack, and that the bomber died.
"We suspect that pro-Taliban militants who are fighting security forces in our tribal areas are behind this attack," an intelligence official said.
Benazir Bhutto, another former prime minister, returned to Pakistan in October after Musharraf agreed to shield her from prosecution in corruption cases she said were politically motivated.
Bhutto's home-coming parade in Karachi was hit by a suicide attack which killed at least 139 people.
She had lost some credibility by holding power-sharing talks with Musharraf before her return, but has become increasingly confrontational towards the general since his imposition of emergency rule.
Musharraf has said emergency rule was imposed in order to tackle the groups that carry out such bombings.
But the president's opponents say that most of those targeted have been political opponents, lawyers and members of the media.
As part of the emergency, Musharraf curbed the media and blacked out private television news channels and international broadcasters, leading to confrontation in the streets between Pakistani journalists and police.
Sabihuddin Ghausi, president of the Karachi Press Club, said: "It is actually a power struggle between the military establishment and the new emerging power centres in civilian life represented by the media barons."
All but the most popular channel, Geo News, have been allowed back on air, though none has continued with the political talk shows that infuriated the government.
Musharraf has until December 1 to swear the oath as a civilian president for another five years after the supreme court, now stacked with judges appointed by Musharraf, confirmed his victory in a presidential poll last month.
He has already promised to resign as army chief. Tariq Azim, the former deputy information minister, said Musharraf's stepping down was likely to happen on Tuesday or Wednesday, re-taking office a day later.
Azim, who is close to the military ruler's circle, predicted the emergency would be lifted before general elections on January 8.
Source: Al Jazeera and agencies