The 57-year-old has pledged to transform Paraguay's impoverished society, where almost half of the of the country's six million people live below the poverty line.
"Today is the end of an exclusive Paraguay, a segregationist Paraguay, a notoriously corrupt Paraguay," he said.
"Today begins the history of a Paraguay whose authorities will be implacable with thieves."
Al Jazeera's Lucia Newman in Asuncion said Lugo's election victory was seen as a chance to create a new political and economic culture after decades of rule by the Colorado party.
Lugo has long-championed the rights of Paraguay's landless peasants and indigenous populations.
One indigenous Indian who travelled hundreds of miles to attend the ceremony told the AP news agency he had high hopes for Lugo.
"I just want him to get rid of the corruption and the inequality so we have a chance at giving my children a future,'' Marcelino Coronel said.
"In the Chaco [region], the government never did anything for us.''
The ceremony was also attended by several Latin American leaders, including Hugo Chavez, the Venezuelan president, and Evo Morales, the Bolivian leader.
Lugo won 40 per cent of the vote in a three-candidate presidential race in April this year.
A newcomer to politics, he was swept to power by a largely grassroots coalition of opposition parties, the Patriotic Alliance for Change, and has the respect of several Colorado party politicians who were disillusioned with Duarte.
The former bishop was given a blessing by Pope Benedict XVI to enter office and a waiver to return to layman's status in order to become president.
On the eve of his inauguration, thousands of supporters, who attended his stadium rally, applauded Lugo when he said he would refuse his presidential salary of about $4,000 a month.
While Lugo has said he will govern for the poor, he has distanced himself from Chavez and his allies, sending a more pro-business message and saying he will reduce rather than increase state control of the economy.
However, the Colorado party, which also supported the 1954 to 1989 dictatorship of Alfredo Stroessner, still dominates most government institutions in the small landlocked country, where corruption is entrenched and just one per cent of the population controls 77 per cent of the land.
One of Lugo's first challenges as president will also be renegotiating hydro-power treaties with wealthy neighbouring nations Brazil and Argentina, in addition to finding a means of helping thousands of landless peasants.
The peasants, who have been seizing private property, have reportedly said they will begin a much larger wave of invasions on land owned by rich farmers as early as this weekend.