Opinion polls have pointed to a slim majority approving the government's plans to extend an agreement with Brussels on the free movement of people to new EU member states, mostly from eastern Europe.
The outcome of the vote is expected by late afternoon on Sunday.
The vote is seen as a test for Switzerland's relations with the EU.
A "no" vote could unravel a number of complex deals with the bloc, aimed at making cross-border trade and law enforcement easier.
With 60% of Swiss exports going to EU countries and 80% of its imports originating from there, the bloc is Switzerland's main trading partner by far, and sour relations would not bode well for its export-dependent economy.
"We shouldn't shut ourselves away. Switzerland is already an island anyway," said bank employee Therese Moser, who voted "yes".
"We shouldn't shut ourselves away. Switzerland is already an island anyway"
In the run-up to the vote, anti-EU campaigners said the modest reforms would lead to higher unemployment in Switzerland's protected labour market - despite the fact that only about 3000 workers from the new EU nations would be allowed in each year.
As in France and the Netherlands, where voters recently rejected a new EU constitution, many Swiss voters have voiced fears of a flood of cheap labour and mass immigration.
Rolf Hoffman, 40, an unemployed gardener, voted "no".
"They will take our jobs and we will not have the money to support them. There will be more people looking for jobs and it will be harder for Swiss to find jobs," he said.
The accord - which already applies to the EU's 15 older states - would allow Swiss citizens to work and settle in the new EU countries, while their citizens in turn could live in Switzerland, provided they have work and can support themselves.
"They will take our jobs and we will not have the money to support them"
"There's room enough. Our people also go abroad, for higher education, or to work. It wouldn't be good if all of them had to return. They should be able to go, especially young people," said Brigitte Eberhard, a pensioner.
Voters in the wealthy alpine state rejected moving towards EU integration in 1992, but have taken steps since then towards closer association.
In June, Switzerland moved a step closer to its European neighbours, voting by a narrow margin to join the EU's passport-free "Schengen" zone.
This referendum is the fourth vote in little more than five years, held on fiercely independent Switzerland's relationship with the 25-nation European Union which surrounds it.