The deal was slammed by pro-Palestinian groups as a retreat from the struggle for equality embraced by President Thabo Mbeki's government.
Olmert and South African Trade Minister Mandisi Mpahlwa hailed the accord as a sign of closer relations during a ceremony held at a conference centre under heavy police guard.
Several protesters representing pro-Palestinian groups, unions and the Movement for Landless People had massed outside the conference centre earlier in the day to denounce the agreement as a sell-out by the ruling African National Congress (ANC) to Israel.
Olmert, the most senior Israeli official to visit South Africa since the end of apartheid a decade ago, was accompanied by 23 influential Israeli businessmen.
He acknowledged that the conflict with the Palestinians was a sore point in South Africa.
"When I go back, I will report on a certain degree of discontent which was expressed on the street but also a much greater sentiment to create a much closer relationship," said Olmert, who is also trade and industry minister.
Waving placards that read "Israel and Zionism Equals Racism" and "Isolate Apartheid Israel", the protesters charged that the ANC government was engaging in double-play by slamming Israel politically but encouraging trade tries for financial gains.
Protesters called for a boycott
and sanctions against Israel
Union of Muslim Students Associations spokesman in South Africa, Muhammad Cajee, told Aljazeera.net that "it's a betrayal of the Palestinian people.
"The ANC has more than historical ties with the Palestinians and, more so, can sympathise with them having overcome apartheid," he said.
Cajee added that "both (Palestinians and South Africans) have a background of fighting oppression and human rights violations".
Earlier on Wednesday, protesters gathered at Johannesburg's main convention centre, highlighting their "discontent and displeasure" at the visit and the agreement.
Naeem Jeenah, spokesman for the Palestine Solidarity Group, told Aljazeera.net: "He [Olmert] is the most racist of all Israeli politicians. When he was mayor of Jerusalem he literally ordered the ethnic cleansing of Palestinians.
"Israel is the last outpost of apartheid in the world. It supported and strengthened ties with the [South African] regime when the world had imposed sanctions".
"The ANC has more than historical ties with the Palestinians and, more so, can sympathise with them having overcome Apartheid"
Union of Muslim Students Associations (UMSA)spokesman
Jeenah added: "In the context of South Africa, with regard to the Bantustans, the international community was convinced that it was a crime against humanity and immoral.
"While in the case of the Israel, the international community is now promoting the same idea to the Palestinians as the solution."
Since it swept to power with the end of apartheid in 1994, the ANC, Africa's oldest liberation movement, has taken a decidedly pro-Palestinian stance, harshly criticising Israel for the separation wall in the West Bank, deemed "the apartheid wall" by South Africans.
But Mbeki's government has been warming to Israel, inviting members of the governing Likud Party to talks in Pretoria last month and offering to share South Africa's experience of reconciliation.
South Africa's main trade union, the Congress of South African Trade Unions (Cosatu), said the investment agreement would "further embolden the Israeli regime and strengthen its determination to defy worldwide condemnations of the brutal policies they have inflicted on the people of Palestine".
Cosatu spokesman Mike Louw said "we are willing to put our money where our mouth is", referring to its call for a boycott of Israel, including approaching companies that deal with Israel.
South Africa's main trade union
criticised the investment pact
Other South African organisations have called for a boycott of Israeli products as well. Among them are the Muslim Judicial Council, Friends of al-Aqsa, the Palestine Solidarity Committee and UMSA.
The coalition of South African non-governmental organisations also said "the visit of a hatemonger like Olmert is not in the best interests of the people of Israel and Palestine. We urge our members to show their displeasure at the visit by joining the protest".
But foreign ministry spokesman Ronnie Mamoepa said the "essence" of the visit was to encourage dialogue between Israel and the Palestinians and not to broaden economic ties.