"This [rocket] attack ... will have unprecedented, far-reaching consequences [and] the Hamas organisation will be the first to feel them," Ehud Olmert said.
The rocket struck a school in the centre of the Israeli coastal city of Ashkelon on Tuesday evening, causing considerable damage but no casualties, the Israeli army said on Wednesday.
The armed wing of Hamas, the Ezzedine al-Qassam Brigades, has claimed responsibility for firing the rocket.
Palestinian fighters frequently fire makeshift rockets into southern Israel. They are generally inaccurate and have a maximum range of about 12km.
However, the rocket fired in Wednesday's attack appeared to have a longer range than most previous ones, bringing the threat of rocket barrages to a large Israeli population centre for the first time.
On Tuesday evening, Israeli aircraft struck the Palestinian interior ministry for the second time in a week.
Rescue workers said at least four people were wounded in the strike and witnesses said a building next to the ministry was hit, causing some damage.
No one was hurt at the university
Hamas controls the ministry, which has nominal authority over some Palestinian security forces.
Israeli aircraft also fired missiles at a Hamas camp in southern Gaza and a Hamas-affiliated university in Gaza City. No one was hurt in either incident.
The attacks came hours after a deadline set by Palestinian fighters holding an Israeli soldier passed, with Israel rejecting demands to release Palestinian prisoners.
Fighters have been holding Corporal Gilad Shalit, 19, since June 25, when he was siezed in a cross-border raid which left two Israeli soldiers and two Palestinian fighters dead.
Israel has responded with more than a week of air strikes and a large military force remains poised on the northern part of the Gaza Strip.