Security was tight on Friday morning for the planned rally at Kabul's National Stadium, with aid agencies advising their foreign employees to stay off the streets.
Mustafa Kazimi, a legislator and former commander, promised there would be no violence.
He said: "It's not a gathering to seek revenge but to promote unity. It's a sign of democracy in Afghanistan."
"It's not a gathering to seek revenge but to promote unity [and] it's a sign of democracy in Afghanistan"
Mustafa Kazimi, Afghan legislator
Zulmay Khan Uriakhail, the deputy police chief, said 12,000 police were deployed across the city and were searching all cars entering Kabul.
He said about 3,000 people had arrived from outside the capital to attend.
Several top commanders who have been accused by some Afghans and rights activists of involvement in war crimes were expected to attend the rally.
Karzai has yet to comment publicly on the proposed amnesty. His spokesman has said the president would not sign anything that goes against Afghanistan's constitution.