Al-Hashemi expressed his appreciation for the visit but asked for a written apology from the US military and "real guarantees" the incident would not be repeated.
Austin also met separately with Nouri al-Maliki, the country's Shia prime minister.
The incident was earlier strongly condemned by al-Hashemi and the Association of Muslim Scholars, which represents many of Iraq's mosques.
"This heinous crime shows the hatred that the leaders and the members of the occupying force have against the Quran and the [Muslim] people," it said.
It added that it held both the US military and Iraqi government responsible for the incident.
The US army earlier said the staff sergeant, who fired bullets at the Quran and wrote graffiti inside it, had already been removed from Iraq and was to be disciplined.
Major-General Jeffrey Hammond, head of US forces in Baghdad, visited leaders of the local community in Radhwaniya, where the soldier fired at the Quran on May 11.
Colonel Bill Buckner, a US military spokesman, said the military viewed the incident "as both serious and deeply troubling," but said it was an "isolated incident and a result of one soldier's actions".