The army and police have been hunting members of a Muslim group after hundreds of its members attacked police stations and government buildings in five northeastern towns last week. State media said they wanted to impose a "Taliban-style" Islamic government.
The seven killed on Monday had sought refuge in Sandiya, a village 100 km from the border with Cameroon, police Commissioner Abd al-Aziz Basiru told reporters.
Basiru said 14 rebels arrived in the village and sought permission to pray. The villagers asked to see the contents of their bags, and when they refused the villagers called the "community vigilantes".
"One of the militants brought out a pistol and shot a villager dead. That provoked the vigilantes who killed seven of them while the other seven escaped with bullet wounds."
The shootings brought the number of confirmed deaths in the week-long uprising to 16.
Basiru said the rebels were "highly placed young men" from all over Nigeria who had abandoned university education to set up strict religious communities in the barren savannah of northeastern Nigeria.
The Guardian newspaper said the group, known variously as Ahlul Sunnah Jamaa, Hijrah and Maitatsine, had about 2000 members and had presented regional authorities with an ultimatum to clean up corruption in Borno state by 27 December.
The rebels are reported to have been "highly placed young men" from all over Nigeria who had abandoned university education to set up strict religious communities
Nigeria's 130 million population is split roughly equally between Muslims and Christians. Religious and ethnic fighting has killed at least 10,000 people since 1999, when military rule ended in the world's seventh biggest oil exporter.