Lugo, who has pledged to clean up corruption and transform Paraguay's impoverished society, where almost half the of the six million population are below the poverty line, saying "it won't be easy, but it's not impossible".
"Today is the end of an exclusive Paraguay, a segregationist Paraguay, a notoriously corrupt Paraguay," he said.
"Today begins the history of a Paraguay whose authorities will be implacable with thieves."
Al Jazeera's Lucia Newman in Asuncion said Lugo's election win had been seen as a chance to create a new political and economic culture after decades of rule by the Colorado party.
Lugo has long championed the rights of Paraguay's landless peasants and indigenous populations.
One indigenous Indian who travelled hundreds of miles to attend the ceremony told AP he had high hopes for Lugo.
"I just want him to get rid of the corruption and the inequality so we have a chance at giving my children a future,'' Marcelino Coronel said.
"In the Chaco [region], the government never did anything for us.''
Lugo won 40 per cent of the vote in a three candidate presidential race in April this year.
A newcomer to politics, he was swept to power by a largely grassroots coalition of opposition parties, the Patriotic Alliance for Change, and has the respect of several Colorado Party lawmakers who were disillusioned with Duarte.
The former bishop was given a blessing by Pope Benedict XVI to enter office and a waiver to return to layman's status in order to become president.
On the eve of his inauguration, thousands of supporters who attended stadium rally, applauded Lugo when he said he would refuse his presidential salary of about $4,000 a month.
While Lugo has said he will govern for the poor, he has distanced himself from Chavez and his allies, sending a more pro-business message and saying he will reduce rather than increase state control of the economy.
However, the Colorado party, which also supported the 1954 to 1989 dictatorship of Alfredo Stroessner, still dominates most government institutions in the small landlocked country, where corruption is entrenched and just one per cent of the population controls 77 per cent of the land.
One of Lugo's first challenges as president will also be renegotiating hydro-power treaties with wealthy neighbouring nations Brazil and Argentina, in addition to finding solutions for thousands of landless peasants.
Landless peasants who have been seizing private property have reportedly said they will begin a much larger wave of invasions on land owned by rich farmers as early as this weekend.