With more than 95% of the votes counted on Sunday, 64% of Brazilians were opposed to the ban, while 36% backed it, said officials at the Supreme Electoral Court.
With a population of 180 million people, Brazil has a death by gunshot rate of nearly 40,000 people a year.
While supporters argued that gun control was the best way to staunch the violence, opponents played on Brazilians' fears that the police cannot protect them.
"I don't like people walking around armed on the street. But since all the bandits have guns, you need to have a gun at home," said taxi driver Mohammed Osei, who voted
against the ban.
The proposal would have prohibited the sale of firearms and ammunition except for police, the military, some security guards, gun collectors and sports shooters.
It would complement a 2003 disarmament law that sharply restricts who can legally purchase firearms and carry guns on the street.
"I don't like people walking around armed on the street. But since all the bandits have guns, you need to have a gun at home"
Mohammed Osei, taxi driver
That law, coupled with a government-sponsored gun buyback programme, has reduced deaths from firearms by about 8% this year, the Health Ministry said.
About 39,000 people in Brazil are killed by guns each year, compared to about 30,000 people in the United States, although the US population is about 100 million more than Brazil's, according to the US Centres for Disease Control and Prevention.
According to Unesco, Brazil ranks second worldwide in deaths by gunshot, with 21.72 per 100,000 people a year. Venezuela ranks higher with 34.3 gun deaths per 100,000.
But in shantytowns such as Rio's Vila do Joao, the rate rises to around 150 per 100,000. And for shantytown males between ages 17 and 24, the death rate is closer to 250 per 100,000.