He was prime minister from 1999 to 2001, a leadership marked by Israel's withdrawal from Lebanon and failed peace talks with the former Palestinian leader, Yasser Arafat.
Jacky Rowland, Al Jazeera's correspondent in Tel Aviv, said: "We don't have official results yet. But the indications are becoming firmer that Ehud Barak has won."
Gil Hoffman, political correspondent at the Jerusalem Post, said that ironically Barak's victory strengthened the coalition government.
"It stabilises the government," he said.
"He wants to be the defence minister under Ehud Olmert for as long as possible. He needs to do that, not for the party, not for the country, but for himself. He needs to repair his image that was tarnished by a premiership that was not successful.
"By this, he could make himself a credible candidate for prime minister against Benjamin Netanyahu in the next election."
The coalition government faces a further day of challenges on Wednesday when Israel holds presidential elections.