Baghdadi Mahmudi replaces former prime minister Shukri Ghanem, who had held the post since 2003.
Mahmudi, a 60-year-old former doctor from the western Libyan town of Sorman, previously served as health minister and deputy prime minister.
The major cabinet reshuffle announced on Sunday saw the creation of seven ministries: Agriculture, transportation, higher education, health, housing, social affairs, and industry and electricity which replaces the former energy ministry.
No reasons were given for the reshuffle in the oil-and-gas rich North African country.
Among the key changes, Finance Minister Mohammed Hwij was named to the post of deputy prime minister, previously held by Mahmudi.
Economy and Trade Minister Abdel Kader Kheir is to be replaced by Taeb al-Safi, and Bakhita al-Sahalawi, a woman, was named social affairs minister.
The last such cabinet reshuffle was in 2004.
Ghanem is no longer part of the cabinet but will head the state-owned Libya National Oil Company, state television said on Sunday.
Ghanem, who studied in the United States, had tried to move the socialist economy towards a free market model, but his policies were criticised by Libya's local People's Committees, which implement government policy.
The committees had previously attacked Ghanem's plans for privatising state companies, freezing salaries and getting rid of subsidies on essential products.
The annual Popular Congress meeting, which serves as the country's parliament, was held in Syrte, 500km west of the capital Tripoli.