Swiss public broadcaster SRF reported, based on a partial count, that the measure was rejected by 61.3 percent of voters, with 38.7 percent in favour.
Around two-thirds of the 2.1 million foreigners living in Switzerland in 2019 were citizens of the EU, as well as Iceland, Norway and Liechtenstein, which with Switzerland are members of the European Free Trade Association (EFTA).
The government had warned that the measure could further strain the rich Alpine's country's deep and lucrative ties to the EU.
That vote threw Swiss-EU relations into disarray, with Brussels warning that any curbs on immigration by EU citizens would put a whole range of bilateral agreements at risk. This time, a more lively vote than usual was expected in the country. Some are dual citizens and wouldn't have been affected by any restrictions.
Sunday's referendum had been initiated by the populist right-wing Swiss People's Party (SVP), which said that current regulations allow for an average of 75,000 European Union citizens per year to enter, leading to overpopulation, rising housing costs and a strained welfare system.
The Swiss Federal Council and parliament have recommended that people vote "no" on the initiative, stating that it would "jeopardise Switzerland's stable relations with its main partner" and would threaten "jobs and prosperity as we face great economic uncertainties".
And the government has cautioned that if Switzerland unilaterally voids the free movement accord, a "guillotine" clause will come into force to freeze the entire package of Swiss-EU deals, including on trade.
Since the last vote, Switzerland has witnessed the personal and professional turmoil that Britain's 2016 referendum to leave the European Union has caused, especially for EU citizens in the United Kingdom and Britons living on the continent.
"I'm not at all in favour, and even more now, because it's not a time to isolate Switzerland", he said.
Elisabeth Lopes agreed, saying: "I'm a daughter of immigrants, so it is a matter that touches me".
Switzerland is set to vote today on five referendum questions including a proposal to end freedom of movement agreements with the EU.
Ending free movement from the European Union is likely to topple other bilateral pacts on land and air transport, procurement, technical barriers to trade, and research.
The freedom-of-movement measure is being considered alongside nationwide votes on paternity leave, tax breaks for child care, purchases of up to 6 billion francs (about $6.5 billion) worth of new fighter planes by 2030, and the right to hunt wolves to keep their population down.
Voter participation Sunday ticked in at almost 59 percent, which is exceptionally high in a country where it is rare to see more than 50 percent of voters turn out for the frequent popular polls.