Olmert has so far resisted the pressure after a government-appointed commission blamed him for serious failings during last year's 34-day war.
Labour, in the midst of its own power struggle, is the largest partner in Olmert's coalition government and its withdrawal could force new elections.
Israel's Haaretz newspaper quoted Olmert's aides as saying they were concerned that members of Labour would push to end their partnership with the prime minister's Kadima party in order to form a new government.
However, some coalition members have suggested that they would stick by Olmert because they might lose significant parliamentary clout if fresh elections were held now.
"It would be best if he resigned and if the country elects a candidate who would reach for peaceful solutions"
Casey, Kitchener, Canada
Send us your views
Amir Peretz, defence minister and head of Labour, is considering stepping down from his post in response to the Lebanon war report.
He has become increasingly unpopular within his own party and may be forced out when Labour holds internal elections on May 28.
Olmert's approval ratings have plummeted into single digits and Tzipi Livni, his foreign miniser and deputy, joined calls for his removal this week.
Miri Eisin, Olmert's spokesperson, said: "The best way to face the challenges, and for the better of Israel, is for the prime minister to stay in his position."
Ofir Pines, a Labour party member, and several other Labour leaders said that Olmert must go, even if it means early elections.
Pines said: "We will make an effort to build a new coalition and a new government. If we won't be able to do so, we will have to have early elections.
"It is not the best option but it's a better option than to stay with the present government."
Danny Yatom, also a candidate for the Labour party leadership, said that as long as Labour stays in the coalition it gives legitimacy to a government that has lost the support of the Israeli people.
Yatom said: "I will try to convince my friends in the faction and in the central committee [to withdraw from the coalition] ... and finally I hope that after such a big demonstration, they will be convinced."
Israel's next general election is not due until 2010.