The crackdown on Thursday came as dozens of Palestinian towns and villages in the West Bank held municipal elections.
Palestinian police chief Ala Husni said that in the wake of Israel's recent pullout from Gaza there is no longer a reason for anyone other than security officers to carry weapons.
"The role of resistance weapons has ended in the streets. They should go back into storage and they should not show up in the streets," he told a news conference.
"Any weapon now in the street is an illegal weapon." He said there were no plans to seize stored weapons.
Interior Ministry spokesman Tawfiq Abu Khoussa said authorities arrested three men carrying Kalashnikov assault rifles on Thursday and confiscated their weapons. Several security officers also were arrested for carrying guns while off duty, he said.
In Washington, State Department spokesman Sean McCormack called the ban "a positive step".
Hamas spokesman Mushir al-Masri said the group would honour the ban on displays of weapons and parades, but ruled out surrendering weapons.
Israeli officials said they wanted to see whether the pledge would be met with results.
"The question that many Israelis have on their minds is whether this is cosmetic or is this a substantial move in the right direction. Of course we very much hope it is the latter," said Foreign Ministry spokesman Mark Regev.
Palestinian President Mahmoud Abbas has urged resistance fighters to give up their weapons. But he has rejected Israeli demands to confront the groups.
In the meantime, Israeli Defence Minister Shaul Mofaz has ordered the current Israeli offensive to continue until at least next week, security officials said.
Israel has carried out airstrikes in Gaza and arrested hundreds of Palestinian men in the West Bank since launching the offensive on Saturday. Officials say Israel must send a message that after the withdrawal, attacks from Gaza will not be tolerated.