His comments came as Israel marked the anniversary of the assassination of Yitzhak Rabin, the former prime minister who was killed by an ultranationalist Jew in 1995 who said he carried out the killing to halt land-for-peace efforts with the Palestinians.
Israeli settlers and security forces clashed for the fourth time in less than two weeks on Saturday, an escalation in violence that Olmert called an "intolerable situation" and prompted the decision to cut off funding to the more than 100 wildcat outposts considered illegal under Israeli law.
But the move would not apply to the more than 120 official settlements.
"Those who strike our children have to know that we won't turn the other cheek"
Itamar Ben Gvir, local settler leader
The international community considers all West Bank settlements to be illegal.
"There is a not insignificant group of outlaws that are behaving in a manner that is threatening the rule of law," Olmert said ahead of a weekly cabinet meeting on Sunday.
"This is an intolerable situation that we refuse to accept."
The decision came after teenage settlers hurled rocks at border police near the West Bank town of Hebron on Saturday, slightly injuring two of them.
A representative from the local settler council said the police had sparked the latest fighting by beating a 10-year-old settler child.
"The child wanted to cross a roadblock... Those who strike our children have to know that we won't turn the other cheek," Itamar Ben Gvir said.
Border police spokesman Moshe Pinchi said he had no knowledge of the alleged beating and accused the settlers of "cynically" sending minors to attack the police.
Ehud Barak, the defence minister, called the violence a "grave phenomenon that no viable society should tolerate".
The European Union condemned the attacks, saying that "it is up to the Israeli government, which has itself condemned these acts, to take the necessary measures to stop them immediately, in accordance with its international obligations".
Settler leaders have also condemned the violence but have made no secret of their community's opposition to the evacuation of settlements that would make way for a future Palestinian state.
Hardline settlers say they have adopted a "price tag" policy of attacking Palestinians or security forces every time an outpost is demolished.
Dov Lior, the head rabbi of the nearby settlement of Kiryat Arba, compared Israeli security forces to "the Nazis in Poland" during World War II.
|The cut in funding does not affect 'legal' settlements in the West Bank [EPA]
"The Nazis also woke people up in the middle of the night and deported them. At that time also we were driven from our homes for no reason other than that we were Jewish," he said.
The government referred directly to the rabbi's remarks in its decision to sever funding to the outposts, saying it would "examine whether state employees are involved in incitement and bring them to justice".
About 100 wildcat outposts dot the West Bank - some consisting of just a few trailers and others of several mobile homes connected to the power grid - usually built as extensions to officially established settlements.
More than 260,000 Israelis live in government-authorised settlements across the West Bank, with another 200,000 in mostly Arab east Jerusalem, which Israel seized and annexed in the 1967 Middle East War.
Palestinians have said the issue of settlements is the main obstacle confronting US-backed peace talks relaunched nearly a year ago.
Olmert has said Israel would have to withdraw from almost all of the territory it captured in 1967.
But he is now serving in a caretaker role until a new government is in place after elections on February 10, after he resigned in September in a corruption scandal, and the peace deal that Israel and the Palestinians had hoped to achieve this year appears out of reach.