The US military confirmed the deaths of the two men.
"The death of these two terrorists is a potentially devastating blow to al Qaeda in Iraq," the US forces in Iraq said in a statement.
The Iraqi government has previously claimed to have captured or killed Baghdadi on several occasions and the US military has even questioned whether he is an actual person or a fabricated figurehead.
"If it proves true this time, it would be a big blow to al-Qaeda; especially as the [Iraqi] government has been blaming al-Qaeda for the spate of bombing attacks that have been rocking Baghdad over recent weeks," Al Jazeera's Zeina Khodr, reporting from Baghdad, said.
"This event is going to boost the reputation of al-Maliki and the Iraqi security forces, which launched several attacks on al-Qaeda hideouts according to the government.
"The government said the computers they seized in those hideouts showed there were communications between al-Masri and al-Baghdadi. It also showed communication with al-Qaeda's overseas leaders including Osama bin Laden."
Baghdadi was last claimed to have been killed last April, but the Islamic State of Iraq denied the claim and said he was "fine", the SITE Intelligence Group said, according a statement released on internet forums on May 11, 2009.
Mustafa al-Ani, a security advisor at Gulf Research Centre, said that he did not believe that Maliki would risk losing credibility by not verifying the identity of the dead.
"It is true we heard many allegations in the past about the capture of Baghdadi and or al-Masri, but they were from the Iraqi side only"
former military intelligence general
"Last year the government showed Baghdadi captured on official TV and then this was denied by the insurgents. I believe Maliki lost credibility as a result," he said.
"So I don't think Maliki is going to risk losing his credibility a second time without verifying the identity.
"At the same time we must ask whether Baghdadi is real. It is a possibility that he is a fictitious character used by al-Qaeda.
"Al-Masri is different - we have photos and a video dating back three years and so his identity is much easier to match."
The timing of the latest operation is vital for al-Maliki who is trying to negotiate support for his State of Law coalition following parliamentary elections in which it emerged as the second largest bloc.
Wafiq al-Samaraei, a former general in the Iraqi military intelligence services and security advisor to the Iraqi president, told Al Jazeera that he believed it was significant that the US military had confirmed the killings.
"It is true we heard many allegations in the past about the capture of Baghdadi and or al-Masri, but they were from the Iraqi side only, and the US army in Iraq used to keep silent," he said.
"This time we have a US confirmation. So, we can say there is some authenticity in today's news.
"In the past, al-Qaeda leaders used to respond to such allegations by issuing either video or audio messages to dismiss the Iraqi government allegations and prove they were still free and operative.
"I believe we need to wait for few days to get the final confirmation, the organisations of the two men are obliged to response either by issuing a recorded message or by announcing the appointment of new leaders."
Joe Biden, the US vice-president, said that the killings were a potentially "devastating" blow to al-Qaeda in Iraq.
"But equally important in my view is this action demonstrates the improved security strength and capacity of Iraqi security forces," he said.
"The Iraqis led this operation, and it was based on intelligence the Iraqi security forces themselves developed following their capture of a senior AQI leader last month."
Biden said an American soldier was killed while supporting the operation and called the soldier, who was not named, a "young hero".