Duterte made the announcement on Tuesday despite a military general saying dozens of fighters, including foreign nationals, were still battling Philippine troops in Marawi.
"I hereby declare Marawi city liberated from the terrorist influence. That marks the beginning of rehabilitation," he said during a visit to the city.
The president did not indicate when he will lift the martial law declaration he issued for the entire island of Mindanao when fighting broke out in May.
Brigadier-General Restituto Padilla, an army spokesman, told reporters the military operation in Marawi would continue "until armed elements are dealt with".
Padilla said between 20 to 30 fighters, who he referred to as "stragglers", remain. It is believed they are holding at least 20 hostages.
Reports said a Malaysian doctor, Mahmud Ahmad - identified as a financial backer of the group - was also on the run.
Al Jazeera correspondent Jamela Alindogan, reporting from Marawi, confirmed there were still "pockets of fighting" in the beleaguered city.
During that operation, about 17 hostages were also reportedly rescued by the military.
Both Abu Sayyaf and Maute had declared allegiance to the Islamic State of Iraq and the Levant (ISIL, also known as ISIS), later joining forces to rampage through the university town in Mindanao.
The five-month battle of Marawi has killed more than 1,000 people and displaced as many as 600,000 while levelling the historic city.
|President Duterte and his defence chief Delfin Lorenzana (with raised arm) rallied troops in Marawi on Tuesday [AFP]|
Richard Heydarian, a political analyst, said the success of the Philippine military could be partly attributed to assistance from US special forces, which provided intelligence as well as equipment during the fighting.
Australia, China, and Russia also provided military hardware to Philippine troops.
"The Philippines can now claim that the threat of an Islamic state is under control," Heydarian said, though he warned fighters could shift to pinpoint attacks on government forces as well as civilians.
While the deaths of Hapilon and Maute and the declared liberation of Marawi dealt a major blow to the rebellion in Mindanao, a retired general criticised the government's handling of the operation.
In an interview with Al Jazeera, the former military official, who asked not to be named, said the fighters "practically held the nation and military hostage for several months".
"The Marawi conflict proved to be a serious challenge to the government and its security apparatus and has grave implications for the Philippines' war against terrorism," he said.
Meanwhile, Marawi residents told Al Jazeera that unless "historical injustices" against the impoverished Muslim communities are righted, the problem will persist.
Alindogan reported it is likely that new leaders will emerge from the armed groups.
"If we look at the history of rebellion in the Philippines, death of leaders does not necessarily mean an end to the presence of these groups," she said.
"This is why peace negotiations with these groups are critical."
|An estimated 1,000 people were killed during the Marawi siege, including 100 government troops [Reuters]|