Germany's ministry of defence said the video was shot in July 2006 at barracks in the northern town of Rendsburg and that the army has known of it since January.
In the video, which was broadcast on national television in Germany and can be seen on internet video sites, the instructor tells the soldier, "You are in the Bronx. A black van is stopping in front of you. Three African-Americans are getting out and they are insulting your mother in the worst ways ... Act."
The soldier fires his machine gun several times while yelling an obscenity in English. The instructor then tells the soldier to curse louder.
New York City officials say they are saddened and frustrated by the video.
City officials said it was disheartening to see such a negative view overseas of the city's Bronx borough, which became a national symbol of urban decay after a 1977 visit by Jimmy Carter, the then US president.
They have called for an apology from the German military.
"Clearly these folks don't know anything about African-Americans or the Bronx"
Adolfo Carrion, Bronx borough president
Adolfo Carrion, Bronx borough president, visited Germany just a few weeks ago to promote tourism to his borough, which recently has cut crime, lowered unemployment and attracted investment.
Of the video, Carrion said: "It really saddens me. The German government obviously has work to do to correct something that is insidious ... Clearly these folks don't know anything about African-Americans or the Bronx."
In Bonn, an armed forces spokesman said the army had launched an internal inquiry into the incident, possibly leading to disciplinary measures against two instructors.
The German army is already being haunted by other scandals.
In the northwestern city of Muenster, 18 German soldiers are on trial for alleged mistreatment of 163 recruits with electrical shocks.
Last year, German soldiers were photographed in Afghanistan, posing with human remains.
An inquiry involving 20 soldiers was launched, and six have already been suspended from duty.