Witnesses said two army truckloads were seen leaving Abu Ghraib prison, west of Baghdad, on Thursday.
Hundreds of Iraqis have been waiting since early in the morning for the release of friends and relatives they hope are part of the amnesty.
As the army trucks left, scores of Iraqis jumped into waiting vehicles to follow the convoy.
US-led occupation forces in Iraq pledged to release around 100 people on Thursday under the programme.
The 100 are the first batch of 500 "low-level security threat" prisoners the US military plans to release in the coming weeks.
The Americans have detained around 9000 Iraqis in the eight months since overthrowing Saddam Hussein.
Many have been rounded up in raids and are suspected of carrying out resistance attacks against US-led forces.
But hundreds of others were detained merely for being in the wrong place at the wrong time.