Ban called for a "full investigation" into the incident to make those responsible for the attack "accountable".
He also announced he was sending a high-level delegation to oversee humanitarian support for Gazans and assist with the rebuilding effort following Israel's three-week offensive.
Ban is the highest-ranking official to visit Gaza since Israel and Hamas declared separate, unilateral ceasefires on Sunday.
Ban stressed "the UN and the international community will support and help you [Gazans] to overcome" the devastation, which he described as "shocking and alarming".
While Ban reiterated his condemnation of Israel's "excessive use of force", he was careful to underline that he believes rocket attacks on Israel by Gazan fighters are also "unacceptable".
Israel had said the aim of its operations in Gaza was to cripple Hamas's ability to launch rockets into the south of the country.
Ban urged Palestinians factions to unite to allow the international community, including the UN, to help broker a lasting peace deal between Israel and the Palestinians.
However, while offering to do "all [he] can as secretary-general of the UN to help in this time of need", Ban is not expected to meet Hamas officials during this visit.
Ayman Mohyeldin, Al Jazeera's correspondent in Gaza, said because much of the international community, including the US and the European Union, regard Hamas as a "terrorist organisation", it would be political difficult for the UN chief to hold direct talks with Hamas officials.
Hamas won parliamentary elections in 2006 which were internationally recognised as being both fair and free.
Rebuilding Gaza's devastated infrastructure is expected to cost billions of US dollars.
John Holmes, the UN humanitarian chief, says hundreds of millions of dollars in emergency aid supplies will be needed for the people of Gaza.
Although water supplies were restored to many Gazan homes as of Sunday, 400,000 were still without running water, Holmes said.
At least 1,300 people killed, including more than 400 children and more than 100 women
At least 5,300 Palestinians injured, including nearly 1,900 children and 800 women
At least 100,000 people forced from their homes
At least 13 Israelis killed, including three civilians
Electricity in Gaza is available for less than half the day and around 100,000 people have been left homeless by the war.
Scores of bodies have been discovered in the rubble of destroyed buildings since both sides declared unilateral ceasefires on Sunday.
Palestinians returning to their neighbourhoods have begun to unearth the true scale of destruction.
Mohyeldin reported that the World Health Organisation had issued a warning of an outbreak of disease as weeks-old bodies remain unburied and sewage continues to flow in many areas because of the destruction of Gaza's infrastructure.
The deposed Hamas-led government in Gaza estimates that more than 5,000 buildings were completely destroyed and another 20,000 damaged or partially destroyed in the fighting.
Despite the three-week Israeli onslaught that killed more than 1,300 Palestinians and destroyed thousands of buildings, Hamas and other Palestinian factions have claimed victory in the fighting.
Following his visit to Gaza, the UN chief also visited the southern Israeli town of Sderot - one of the main targets for Palestinian rocket-launching squads based in Gaza.
Ban said he was "totally committed to Israel securing its rightful place as a nation within secure and recognised borders" and called for the current ceasefires to be turned into a durable truce "respected fully by all parties concerned".