Lower house speaker Somnath Chatterjee interrupted parliament to announce Rao's death on Thursday and called for two minutes of silence.
"We express our deepest condolence," Chatterjee said.
The 83-year-old, who was hospitalised on 8 December, launched socialist India on the path of free-market reforms in 1991 after bringing in the present prime minister, Manmohan Singh, to head the finance ministry.
After the assassination of former prime minister Rajiv Gandhi in 1991, Rao was chosen to lead his Congress party out of crisis. He became prime minister later that year.
"It's a great loss. He made an enriching contribution to Indian politics," Congress Party spokesman Anand Sharma said.
Rao was elected to the federal parliament in 1977 and held several cabinet posts, including foreign minister under both Indira and Rajiv Gandhi.
"It's a great loss. He made an enriching contribution to Indian politics"
Anand Sharma, Congress Party spokesman
Rao became prime minister in 1991 with India on the edge of bankruptcy.
When he stepped down as leader of the world's largest democracy in 1996, the economy was back on the growth track, but the Congress was in turmoil following its poorest performance at the ballot box since independence from Britain in 1947.
His defeat paved the way for the rise of the Hindu nationalist Bharatiya Janata Party and splinter regional parties made up of former Congress members.
"It's a sad news. It's shocking news," M Venkaiah Naidu, former BJP president, said. "He has contributed to the development of the country and his sudden demise is a great loss to the nation."
"It's a sad news. It's shocking news. He has contributed to the development of the country and his sudden demise is a great loss
to the nation"
M Venkaiah Naidu
former BJP president
A member of the Brahmin caste from southern India, Rao was in the twilight of a political career spent in the shadow of the Nehru-Gandhi dynasty when he was called upon to take the helm of the government.
Rao, who had just undergone heart surgery and was preparing to bow out of public life, assumed the reins of the party and the government after Rajiv's widow and now Congress president, Sonia, turned down an offer to step into her slain husband's shoes.
In his first months after taking power in June 1991, Pamulaparti Venkata Narasimha Rao, "PV" to his friends, confounded the sceptics and placed India firmly on a path of ambitious economic liberalisation.
He named former World Bank economist Manmohan Singh as finance minister and gave his blessing to a radical U-turn from the inward-looking quasi-socialist policies that had been set by Jawaharlal Nehru after independence from Britain.