The cases, detailed on Monday by the department's inspector general, Glenn Fine, include accusations of use of excessive force, verbal abuse, religious insensitivity and illegal searches by the Federal Bureau of Investigation.
A total of 1,073 civil rights complaints were received by US authorities from people imprisoned under the Patriot Act adopted in the wake of the September 11, 2001 attacks on the United States, according to the document.
However, Fine deemed only 34 of the complaints "credible," according to the report which was made public on Monday, and which doesn't mention names or locations.
Under investigation are allegations by approximately 20 inmates that a prison guard verbally abused a Muslim inmate and ordered him to remove his shirt so the officer could use it to shine his shoes.
Also being probed is a charge by an unnamed Egyptian national arrested by the FBI immediately following the September 11 attacks who insists he was submitted to multiple and repeated body cavity searches, denied access to legal counsel and the Egyptian consulate and forced to consume food prohibited by his religion.
Several FBI agents are being investigated following allegations that they conducted an illegal search of an Arab-American's apartment, vandalised it, stole items, called the complainant a terrorist and later, having failed to find any evidence of his terrorist connections, tried to frame him on drug charges, according to the report.
An immigration officer is accused of holding a loaded gun to an foreign detainee's head, while a prison doctor allegedly told a Muslim patient: "If I was in charge, I would execute every one of you ... because of the crimes you all did."
And the newly created Department of Homeland Security is investigating an employee accused of mistreating a Muslim visitor by asking him in front of others if he "wanted to kill Christians and Jews."
The American Civil Liberties Union expressed concerns that so few of the complaints had led to formal investigations.
"What remains most disturbing is the tendency of the Justice Department to minimise and downplay what were serious civil liberties and civil rights violations," said Laura Murphy, head of the ACLU Washington office.