The footage appeared to show soldiers dragging youths and kicking them.
A military spokesman in Iraq condemned "all acts of abuse and brutality" by British troops, saying the allegations related to a "tiny number" of soldiers.
The Royal Military Police are probing the incident, which the newspaper said happened in southern Iraq in 2004.
On the tape, described as a "secret home video", an unidentified cameraman is heard laughing and egging his colleagues on. It was apparently filmed for fun by a corporal.
The Ministry of Defence said it was aware of the allegations.
Flight-Lieutenant Chris Thomas, the British military spokesman, based in Basra, said: "We hope that the good relations that the multinational forces have worked very hard to develop won't be adversely affected by this material."
He said the newspaper's claims related "to only a tiny number of the 80,000 personnel that have served in Iraq".
The tape is said to show a disturbance in the street outside what the paper calls a military compound.
Soldiers are seen chasing youths involved in the disturbance, dragging four of them into the compound and beating them on various parts of the body with batons and kicking them, one in the genitals.
The News of the World said a soldier could also be seen kicking a dead Iraqi in the face.
The paper claims that it has established that the soldiers involved were British, but would not disclose which unit or regiment were allegedly involved.
It said it has made exhaustive checks to establish the video's authenticity after obtaining the footage from a whistleblower, who they declined to identify.
Stuart Kuttner, managing editor of the News of the World, said: "We've made inquires of the source, of people around the source, of military experts, of the Ministry of Defence and beyond.
"Given that there was slip-up by a newspaper some time ago, we've satisfied ourselves that this an absolutely genuine article."
The Ministry of Defence said it took any allegations of abuse very seriously.