Sheikh Abdul Aziz al-Humain was appointed as the new head of the Commission for the Promotion of Virtue and the Prevention of Vice which controls the religious police, replacing Sheikh Ibrahim al-Ghaith.
The police have wide powers to search for alcohol, drugs and prostitution, ensure shops are closed during prayer times and maintain the system of sexual segregation in Saudi society.
The reshuffle, King Abdullah's first since he took power following the the death of his half-brother in 2005, also saw new education, justice and information ministers appointed.
"This is a turning point. It is the biggest change that happened in this country in 20 years," Mohammad al-Zulfa, a member of Saudi Arabia's Shura council, told the AFP news agency.
"It is a new start for King Abdullah. People are expecting changes," he said. "These are new faces who can bring change."
The monarch also appointed Abdul-Aziz Khoja, who was previously ambassador to Lebanon, as information minister, replacing Iyad bin Amin Madani, state-run al-Ekhbariya television reported.
Clerics had often criticised Madani for allowing the local press to take greater liberty in challenging the establishment.