Officials said that al-Jaafari had won 64 votes on Sunday, narrowly defeating Adel Abdul Mahdi, the vice-president, who got 63 in a ballot after the group failed to reach an agreement by consensus on Saturday.
According to the Iraqi constitution, the new president will formally designate the choice of the biggest bloc in parliament after the assembly convenes.
Shia lawyers cast their votes at the heavily guarded home of Abdul-Aziz al-Hakim, the head of Abdul Mahdi's party.
The contest for prime minister became a two-horse race after Nadim Jabiri of the Fadhila party and Hussein Shahristani from an independent Shia bloc decided to withdraw.
Al-Jaafari, a doctor, is a member of the Dawa party and spent years in exile in Iran and Britain before returning to his homeland after the US-led coalition ousted Saddam Hussein in 2003.
His government, which took office in April, had been widely criticised for failing to improve the country's crumbling infrastructure or deal effectively with security.
The alliance of Shia religious parties took the biggest number of seats in the vote on 15 December.
The United Iraqi Alliance was confirmed on Friday as the winner of the election in December, paving the way for the formation of the first permanent post-Saddam government.
It won 128 seats in the 275-member parliament, while an alliance of Sunni and secular groups, the Joint Council for National Action, took 80, and the Kurdish Alliance won 53.
The remaining seats are shared by small parties, mostly representing ethnic and religious minorities.
Meanwhile, there was no let-up in violence as bomb blasts and gun attacks killed at least five people and wounded at least 38 in Baghdad and outside the Iraqi capital on Sunday.
A boy looks from his house at the
site of a bomb in Baghdad
Two Iraqi police officers were killed and another nine wounded when unidentified gunmen attacked a police convoy near Amiriyat al-Falluja in the city of Falluja, an Iraqi police source told Aljazeera.
Iraqi police sources said seven police officers were wounded when two explosive devices targeting their patrol blew up in Kirkuk in northern Iraq.
In central Baghdad, 13 people, including three police officers, were also wounded when two explosive devices blew up.
In al-Aazamiya, police sources said a booby-trapped bicycle blew up near a US patrol, but no casualties were reported.
The US military said insurgents fired a mortar into Baghdad's heavily fortified Green Zone, home to the US embassy and Iraqi government, causing an explosion that rocked Baghdad but caused no casualties.