There were conflicting reports about the number of casualties with some police sources saying at least 10 people were killed and eight injured.
The Reuters news agency, however, said other police sources reported that only three people had been wounded.
The two Katyusha rockets slammed into the Abu Dishir district, which is surrounded by a predominantly Sunni neighbourhood.
Mosque targeted
It was not clear whether the Green Zone blast was intended as a signal to the Iranians.
The explosion ripped through the street at 08:45 am (0545 GMT) during the busiest period of the morning rush hour, when nearby roads were packed with motorists and pedestrians heading for work, many in nearby Iraqi ministries.
Khalil Saadati, an Iranian diplomat, said: "It was close to the embassy, but we weren't the target."
On Saturday, a fuel tanker rigged with explosives killed 40 people when it blew up near a Sunni mosque in western Iraq .
The blast came a day after the mosque's imam had criticised al-Qaeda fighters, police and residents said.
The bomb exploded in a market in the town of Habaniya in the province of Anbar, where US forces are battling armed Sunni Arab groups.
Local police said they believed the mosque was the target, adding that the market had been destroyed and 64 people wounded.
Women and children were among the dead, they said.