The decision was revealed by Lieutenant-General Dan Halutz at a closed meeting of the Foreign Affairs and Defence Committee at the Israeli parliament on Tuesday.
According to a participant in the meeting, Halutz told the committee that unless the Palestinians implement their part of the crossings agreement, preventing fighters and weapons from entering Gaza through Rafah, "then we cannot allow other places to be breached and open as well".
The Haaretz daily reported Halutz also saying the passage would not be opened as long as Palestinian resistance fighters in Gaza fire rockets at Israel.
The accord, including opening Rafah, other crossings and the "safe passage" through Israel, was hammered out by US Secretary of State Condoleezza Rice last month.
Israel is concerned that weapons entering Gaza could be transferred to the West Bank.
Palestinians and European monitors at Rafah say the problems there are only technical.