Two others removed in Monday's cabinet shake-up were Jesus Reyes Heroles, the head of state energy company Pemex, and Alberto Cardenas, the agriculture minister.
Calderon, whose term runs to late 2012, gave no explanation for the three resignations.
Medina-Mora is the architect of the government's anti-drug campaign, which Calderon had staked his presidency on since taking office in 2006, but which has so far failed to defeat powerful cartels.
Al Jazeera's Franc Contreras, reporting from Mexico City, said the shake-up appeared to be an admission by Calderon that the policy against drug cartels was not working, especially after the latest violence last weekend where key politicians were killed.
|Drug-related violence remains high despite a country-wide crackdown on cartels [EPA]
In July voters dealt the president's National Action party a defeat in the country's mid-term elections and the shake-up appeared to be an attempt to work with the opposition PRI party, our correspondent said.
"What we are looking at here is a major shake-up; Calderon is signalling to Mexico and the rest of the world that he is ready to do something different to tackle the drug cartels," he added.
More than 13,000 people have been killed in drug-related violence and turf wars since 2006, despite a crackdown by thousands of troops and federal police.
A large deployment of troops in the city of Ciudad Juarez, on the border with Texas, has not slowed a wave of drug killings there.
Medina-Mora did not give a reason for his resignation, saying in his farewell speech that "we have done a lot to clean the house", referring to his efforts to combat rampant police and government corruption.
The outgoing attorney general admitted "successes and errors like in all human endeavours" but defended Calderon's tough campaign against Mexican drug gangs.
"The strategy for recovering the public's security and the tactic of changing the way things were, have been correct," he said at a ceremony announcing the cabinet changes.
"Progress has unquestionably been made. The historic decision to use all the power of the state to put a stop to the power of the criminal organisations was fundamental to ensure our future as a nation."