Ronnie Bar-On said on Monday that he had sent letters to the four giving them 30 days to decide.
"I told them resign or you will no longer be among us. They've got 30 days to make up their minds," Bar-On said on Israel's Channel 2 television.
The four members targeted by the ultimatum are Khaled Abu Arafeh, the Jerusalem affairs minister and members of parliament Muhammad Abu Teir, Ahmed Attun and Muhammad Totah.
The four, who were summoned to Jerusalem's central police station on Monday evening to hear the Israeli ultimatum to leave Hamas or quit from their parliamentary posts, or be kicked out of East Jerusalem, refused to sign the letters.
Abu Arafeh condemned the move.
"Withdrawal of residency cards would constitute a new crime by the occupier in its efforts to make Jerusalem Jewish and ethnically cleanse its Palestinian inhabitants," he said.
The Israeli government said in April that it was withdrawing the Israeli residency cards of the Hamas members of parliament.
Israel gives residency to Palestinian residents of East Jerusalem which affords them some rights. But these limited rights are not given to fellow Palestinians who live in Gaza and the occupied West Bank.
Abu Arafeh, Attun and Abu Teir were detained by Israeli police last week for questioning on suspicion of organising a protest at the al-Aqsa mosque compound.
Israel bans all Palestinian political activity in East Jerusalem which was occupied and annexed - a move not recognised internationally - following the June 1967 Middle East war.
The Palestinians want to establish the capital of their promised future state in East Jerusalem.