Two-thirds of Ghajar, at the foot of Mount Hermon, is on Lebanese territory while the other third is in the Israeli-occupied Golan Heights.
A spokesman for the Israeli army said that soldiers on patrol had initially been shot at from the Lebanese side of the border.
"Shots were fired from Ghajar against an army force. The troops returned fire and identified hitting one gunman," he said.
The Israeli military said that three drug smugglers were involved in the shooting.
A Lebanese army spokesman said that the men shot by the Israelis were unarmed.
A curfew was subsequently imposed on the Israeli-controlled side of Ghajar.
The peace settlement at the end of the 2006 war between Israel and Hezbollah has seen a 13,000-strong UN Interim Force in Lebanon (Unifil) placed along the Lebanese-Israeli border.
Unifil confirmed on Sunday in a statement a "shooting incident in the area of Ghajar" and said it has launched an "immediate investigation to ascertain the facts, looking into initial allegations of smuggling".
Yasmina Bouziane, a Unifil spokeswoman, said a Lebanese man was evacuated by Unifil troops to a hospital in Marjayoun where he was later declared dead.
Another individual was evacuated by the Lebanese army, the statement said.
Bouziane said Major-General Claudio Graziano of Italy, Unifil's commander, was "in contact with senior officers on both sides, urging them to show maximum restraint".