The suspect, named as Boyungan Bongcac, grabbed a rifle from a guard and took a policeman hostage whilst in detention at police headquarters, in an apparent escape attempt on Tuesday.
The man was trapped in an office and surrounded by members of the elite Special Action Force, who eventually shot and killed the suspected rebel after a three-hour standoff.
One police hostage was shot in the head while two officers who came to his aid were also killed as they stormed the building, police spokesmen said. The other officers were reported wounded.
Bongcac had been arrested on suspicion of having participated in a deadly bombing in the southern city of Zamboanga on 18 October, 2002.
The incident is the second major embarassment caused to police by the Abu Sayyaf this year.
In July, two Abu Sayyaf members, along with Jemaa Islamiyah bomb expert Fathur Rohman al-Ghozi, escaped from a jail in the police headquarters.
President Gloria Arroyo is currently in Bali, Indonesia, for a summit of the Association of Southeast Asian Nations to discuss anti-terror measures.
Abu Sayyaf rebels have been blamed for bombings and kidnappings in the southern Philippines for the past 10 years. They have also been accused by both Washington and Manila of having links to al-Qaida.