According to police, the suspect had been running a water-sports company at a luxury hotel.
Mohammed Adow, Al Jazeera's correspondent in Mombasa, said staff told him police gave a surprise visit to the watersports club on Thursday afternoon, and "whisked their manager away".
"The police say they are acting on a tip-off, because this man has been here at the watersports club for a long time. He's never been suspected of anything like this," he said.
Ratko Mladic, a former Bosnian Serb general, and Goran Hadzic, a former Croatian Serb leader, are the two remaining fugitives wanted by the International Criminal Tribunal for the former Yugoslavia.
A Serbian source has told the AFP news agency that the suspect was not Mladic or Hadzic, but a German citizen of Yugoslav origin.
Mladic is wanted for genocide over the 1995 Srebrenica massacre, where about 8,000 Muslim men were killed.
Hadzic is wanted in connection with atrocities during the war in Croatia.