The offensive comes a week after seven Filippino hostages captured by the Abu Sayyaf were beheaded.
The decapitated remains of the construction workers, including two teenagers, were delivered to army bases on Jolo.
The situation on Jolo, scene of years of clashes between the government and Abu Sayyaf, recently became more complicated after Habier Malik, a commander with the Moro National Liberation Front (MNLF), turned his guns on troops despite a 1996 peace agreement.
The military responded by capturing two of Malik's camps and scattering his 300-400 armed followers, who officials said took shelter with remaining Abu Sayyaf fighters.
The MNLF, one of three rebel groups in the country, signed a peace deal with the Philippine government in 1996.
Malik and the MNLF had been helping government troops and their US advisers flush out members of the Abu Sayyaf and regional terror network Jemaah Islamiah from Jolo's forests.
The military, which had already launched airstrikes against Malik, triggering mass evacuations and criticism from Muslim nations, vowed to step up its offensive against the Abu Sayyaf.
But analysts fear that the military's fierce campaign against the MNLF could disrupt local support for its campaign against the Abu Sayyaf and bury any hope of salvaging the 1996 peace deal.
It could also foster increased cooperation between the MNLF and the Abu Sayyaf, they say.