No reason was given for the police move, which came ahead of a deadline of 2pm on Wednesday for Anwar to voluntarily present himself for questioning over the allegations.
Speaking to Al Jazeera, Syed Hamid Albar, Malaysia's home affairs minister, said Anwar had "been arrested according to due process" and was not being treated differently from any other suspect.
"Now it's up to the police and the prosecution side to decide the next course of action," he said.
Anwar's arrest is expected to deepen political divisions and tensions that have been running high in Malaysia since the governing coalition lost its parliamentary two-thirds majority in the March general elections.
"There is no basis for this whole fabrication and malicious attacks"
Opposition legislators including Wan Azizah Wan Ismail, Anwar's wife and leader of the opposition Keadilan party, had gathered outside police headquarters in anticipation of his arrest.
Wan Azizah said her husband, who is officially an adviser to Keadilan, called her on her mobile phone shortly after being arrested.
"I feel apprehensive because my husband… is not that well," she said.
"He has a bad back, he's had surgery. And [during] the brief conversation he said they [the police] were not gentle."
Earlier on Wednesday Anwar gave a statement to the country's corruption watchdog over reports he had lodged against the national police chief and the attorney-general.
"There is no basis for this whole fabrication and malicious attacks," he said. "It is just a repeat of the 1998 script. You can see the pattern."
Anwar claims the police chief and attorney-general had fabricated evidence in the investigation into his police detention in 1999, during which he was allegedly beaten up.
Speaking to reporters in Putrajaya before his arrest he said planned to give his "fullest co-operation" to police and would deliver himself voluntarily to police headquarters at the 2pm deadline.
Azmin Ali, vice-president of Keadilan, said the arrest was "certainly outrageous and very uncivilised" especially after Anwar had made an appointment with the police.
"This is not a criminal case but a political case," he added.
In 1998 Anwar faced similar charges when he was accused of sodomising a male assistant.
The accusation led to his sacking from government posts of finance minister and deputy prime minister and an eventual jail sentence.
His conviction on sodomy charges was later overturned by Malaysia's high court, but a related corruption conviction remains.
That case sparked massive street protests for several weeks.