The accounts on Thursday came a week after forces loyal to the government of Idriss Deby repelled an attack on the capital by rebels of the Chadian United Front for Change (FUC).
"Since the attack, men in uniform are hunting rebels in hiding, but innocent people, who were forced to receive the FUC rebels in their homes as they were fleeing, are taken aside and beaten by soldiers," said Massalbaye Tenebaye, president of the Chadian Human Rights League (LTDH).
"It's a witch hunt," he added, citing other cases among dozens of civilians he says have been arrested.
Tenebaye said that security forces were meting out "collective punishments", particularly to ethnic Tamas and Arabs who are targeted because they are from the same ethnic groups as the rebels.
"Since the attack, men in uniform are hunting rebels in hiding, but innocent people, who were forced to receive the FUC rebels in their homes as they were fleeing, are taken aside and beaten by soldiers"
The rebels, based on the eastern border with Sudan, had marched on N'Djamena on April 13 to try to put an end to Deby's 16-year-rule.
They also wanted to disrupt a presidential election scheduled for May 3 in which he is standing for a third term. The main opposition parties have said they would boycott the vote.
A young tradesman who asked not to be named said he saw police break into a shop in N'Djamena and arrest four of his cousins.
"The police called their names, one by one. The four brothers did not put up any resistance and the police took them away. We don't even know where they are now," he said.
The police gave no reason for the arrests but shortly after last week's fighting, another brother of the same family had been named on national television as a member of the FUC, the witness said.
"The police probably think they were accomplices of the rebels. But they themselves learned from the television that their brother had joined the FUC," the tradesman said.
Police arrested the imam of a town centre mosque because a fleeing rebel had passed through his house, and a mechanic was arrested shortly after the attack and named as a "prisoner of war", said Tenebaye.
Abderamane Djasnabaille, Chad's human rights minister, acknowledged there were instances of what he called "overflow" to civilians in about 20 arrests aimed at rebel fighters. But he denied a witch hunt.
Rebels wanted to overthrow Deby,
who has been in power 16 years
"Since there have been armed confrontations, that reduces the field of rights and personal freedoms," he said.
"The rebels benefited from complicity in their attack on N'Djamena, so it is normal that the security services carry out enquiries," Djasnabaille said.
"These enquiries do not deliberately target Tamas or Arabs, but in practice they lead each time to people of these communities."
The Rights League was trying to collect testimonies, a task made difficult since relatives and neighbours "are traumatised ... afraid to talk," Tenebaye said.
Some have left their homes in fear of reprisals.
"We are afraid," said the cousin of the four arrested brothers. "This time, it was them. Next time, maybe it will be us."