Umar, 69, had been battling Hodgkin's disease, a cancer of the lymph nodes, for more than a year.
He died on Saturday morning at a hospital in Cape Town, the government said in a statement.
"A patriot has gone to rest, forever," the statement said. "A stalwart has breathed his last breath at his post."
A leading member of the ruling African National Congress, Umar was a spokesman for former President Nelson Mandela in the months leading to his release after 27 years in jail.
He then participated in the negotiations that paved the way for South Africa's first all-race elections in 1994.
As justice minister in Mandela's government, he oversaw the establishment of the landmark Truth and Reconciliation Commission that heard victims' accounts of gross human rights abuses and offered the perpetrators amnesty in exchange for a full account of their crimes.
"A patriot has gone to rest, forever ... a
stalwart has breathed his last breath at his post"
South African government
"South Africa needs to make a break from the past," he said at the time.
"This is our one hope of turning South Africa around from the path of violence and intolerance."
Umar was appointed to his current position when President Thabo Mbeki succeeded Mandela in 1999.
Born in Cape Town in 1934, Umar studied law and set up his own legal practice in 1960 because he could not find a firm that would employ black lawyers.
Mandela oversaw the appointment
of Umar as Justice Minister in 1994
Throughout his legal career, he defended those who suffered under apartheid's racist laws.
He represented trade unionists and political prisoners, including some of those serving sentences at the notorious Robben Island prison where Mandela spent most of his years in jail.
Umar also led the Western Cape region of the United Democratic Front, the ANC's voice in South Africa when it was banned and forced into exile.
Banned and detained
For his efforts, he was harassed, banned and detained by the apartheid regime's security forces. Members of a military death squad even plotted to assassinate him by tampering with his heart medication.
When Mandela was elected South Africa's first black president in 1994, Umar was appointed justice minister in a government of national unity, serving with his former foes.
"South Africa needs to make a break from the past...this is our one hope of turning South Africa around from the path of violence and intolerance"
Abd Allah Umar,
First ANC-appointed South African Justice Minister
Praised for the reforms that he brought to South Africa's once oppressive justice system, he had drawn criticism in recent years for the country's high number of road fatalities.
After he was diagnosed with cancer in January 2003, he withdrew from most government functions. But officials said he kept in touch with the department.
Umar was honoured with awards in Chile and Germany for his contribution to the struggle for human rights in South Africa.
Umar, who is survived by his wife, Farida, and three children, will be buried on Saturday. He will be given an official funeral with "all the honours due to a serving member of cabinet", a government spokesperson said.