A suicide attack on Monday in the southern Israeli city of Dimona killed one woman and two bombers.
Responsibility for the operation was claimed on Wednesday by Hamas, which said the bombers entered Israel from the West Bank and not from Gaza via Egypt as originally claimed.
Speaking on the Israeli proposal, the officials said that because of the mountainous terrain, the fence will not extend along the entire 230km border, but will be augmented by sensors.
The decision was taken at a confidential meeting where security officials briefed the security cabinet, comprising Olmert, Ehud Barak, the defence minister, and Tzipi Livni, the foreign minister.
The fence was originally proposed years ago, but was never built because of the cost, estimated at between one billion and two billion shekels ($270m).
On January 23, fighters affiliated with Gaza's ruling Hamas demolished sections of the Rafah border wall using explosives.
Egypt closed the border on Sunday to Palestinians seeking entry, but is allowing Gaza residents to return to the enclave in controlled batches.
The Israeli security cabinet's dismissal of the proposal for bolstering the Egyptian military presence along the Gaza border, amounted to a rejection of the advice of its own bureaucrats.
The foreign ministry had recommended approval for Egypt to double the number of its guards at the border to 1,500.
Under an Israeli-Egyptian peace agreement, the number of troops that can be deployed along the frontier is limited.
Egypt had no comment on the Israeli decisions.
Ahmed Abu Gheit, the foreign minister, called on Tuesday for reason to prevail within the leadership of Hamas in Gaza, after Palestinians and Egyptian forces exchanged fire at the border, killing one person and wounding 59 others.
"We ask the authorities who control the Gaza Strip to allow the return of EU observers and Palestinian Authority police," Abu Gheit said.