The two Norwegians and two Ukrainians were kidnapped from the vessel, owned by Norwegian shipping company Trico Supply, late on Wednesday as it was carrying supplies to an oil rig off Nigeria, company spokesman Bjorn Endresen said.
Hafiz Ringim, the police commissioner for Bayelsa state, where the attack happened, said: "They were working in their boat around 4 or 5am when some armed men believed to be disgruntled members of the community attacked them and took them away."
"Right now, we have not been able to make contact with the hostages, but we are working on that."
He said 11 other crew members were aboard the ship at the time of the kidnapping late on Tuesday, but did not give further details.
Eight foreign workers were kidnapped from the same rig for two days in June in a dispute with a nearby community over jobs and investment.
Also on Wednesday, militants who kidnapped a German contractor last week issued a photograph of him in an email statement which restated their demands for the release of two jailed ethnic Ijaw leaders.
The email included a statement from the captive, who gave his name as Guido Schiffarth, which said he was being treated well but wanted to go home.
Norway's ambassador to Nigeria, Tore Nedreboe, told Reuters that officials had indications of who was behind the abductions of his countrymen and hoped to get into contact with them soon.
"We are hoping that this will be solved fairly fast. There is hope of getting in contact with the kidnappers relatively quickly and getting into negotiations with them," he said.
Militancy is fuelled by widespread feelings of injustice in the vast wetlands region where most people live in poverty despite the wealth being pumped from their country.
A series of attacks by the Movement for the Emancipation of the Niger Delta (MEND) in February forced Royal Dutch Shell to evacuate hundreds of staff from the western delta, reducing output by about 500,000 barrels a day.
The kidnappings and other attacks on Nigerian oil installations have helped push up world oil prices.