So far authorities have found no trace of the hostages or word from the kidnappers amid fears that the hostages may be used to demand for ransom.
The Red Cross said they have also not received any word from the kidnappers.
"We have not heard anything from anybody," Roland Bigler, a Red Cross spokesman, told a local radio station.
"We don't know what group was behind the kidnapping."
Marine Lieutenant-General Nelson Allaga, the military commander in the western Mindanao area, said no group has come forward to claim responsibility for the abduction.
|The Red Cross workers were abducted while heading to Jolo airport [Reuters]
He however said they had information that suggested a former guard at the provincial jail the Red Cross team had visited earlier this week could have been involved in the abduction.
"We're still validating information a former jail guard was in cahoots with the gunmen," Allaga said, adding that the suspect was also believed to have been involved in the escape of 12 inmates from the prison on Tuesday.
Jolo police said the abduction was led by Albader Parad, a top leader of the Abu Sayyaf group.
Julasirim Kasim, the island's police chief, said authorities are investigating links between the kidnapping and Tuesday's jailbreak involving some escapees allegedly linked to the group.
Norberto Gonzales, the Philippines national security advisor, told reporters that the rescue operations were "on full blast", and that the safety of the hostages will be given the "utmost priority".
Fighters from the Moro Islamic Liberation Front (MILF) have said they would help locate and work for the release of the captives.
"We've asked our political and military people on the ground to gather information and help for the safe release of the Red Cross team," Mohaqher Iqbal, a senior rebel leader, told Reuters.
He said the MILF condemned the kidnapping because the Red Cross was a neutral organisation.
The last high-profile abduction in the area was of Ces Drilon, a senior anchorwoman with the Philippines' ABS-CBN television network, her cameraman and an academic on June 8 last year.
They were released 10 days after being kidnapped by suspected Abu Sayyaf fighters.
The kidnappers reportedly demanded $1.12m for the hostages but police said no ransom was paid.