Yasuhisa Shiozaki, the chief cabinet secretary, said the government would strengthen co-operation among law enforcement agencies to stop gangsters from getting their hands on guns.
Shinzo Abe, the prime minister, has described the murder as a "threat to democracy" and called for a full investigation.
Ito, a three-term mayor of the city, was in the midst of campaigning for re-election.
Shootings are rare in Japan, and organised crime groups are said to be behind most of the relatively few that do occur.
Regulations on handgun ownership are among the tightest in the world.
According to police, there are about there are about 84,500 gangsters across Japan, while two-thirds of the country's 53 known shootings last year were gang-related.
|Police say a senior gang member has confessed |
to carrying out the shooting[Reuters]
Police said they are holding Tetsuya Shiroo, an alleged senior member of the Yamaguchi-gumi crime sydicate, in custody over the killing.
Early news reports speculated that Shiro-o killed Mayor Ito because he was angry about the city's refusal to pay him for damage done to his car at a public works site.
He tried unsuccessfully to get compensation from the city after his insurance company refused to pay up, according to Japanese broadcaster NHK.
But Al Jazeera correspondent David Hawkins, who is in Japan, says the killing is looking more and more like a political mob hit.
He says police are trying to confirm reports that the mayor was involved in a dispute with the Yakuza over city construction contracts.
Tuesdays' shooting was the second attack in the last 20 years against a mayor of Nagasaki.
In 1990, Hitoshi Motoshima, the mayor then, was shot and seriously wounded by a right-wing activist after saying that Japan's emperor bore some responsibility for World War II.
Backed by the ruling Liberal Democratic Party, Ito was campaigning for his fourth term in office in Sunday's elections.
He was born in Nagasaki on August 23, 1945, just two weeks after the US dropped an atomic bomb that devastated the coastal city shortly before the end of World War Two.
He is survived by his wife and two daughters.