"The UN must be given immediate unhindered access to the camps, the overcrowding must end and there should be special programmes for pregnant mothers and children," Ban said after meeting Rajapakse in the central town of Kandy.
As he toured the sprawling Manik Farm complex, which is home to most of the 300,000 people forced to flee their homes during the conflict, Ban heard complaints about shortages of food, water, sanitation and medicine.
Residents of the camp are living in tents or makeshift buildings constructed from corrugated iron.
Chicken pox and skin diseases are sweeping through the camp and hepatitis is a growing problem because of poor sanitation, the Oxfam aid agency said.
"I'm very moved after what I have seen. I've seen so many wounded," Ban said after spending 20 minutes walking through the camp.
"There are huge challenges that can only be overcome by strong support from the international community."
International aid organisations, including the Red Cross and UN agencies, have complained that the government has restricted vehicle access to the camp, making it difficult to address the needs of the residents.
"We have access, but we don't have free access for vehicles and that is posing some problems," John Holmes, the UN humanitarian affairs chief, told Al Jazeera.
"There are concerns about the number of vehicles circulating and we are trying to resolve those ... we need to otherwise we won't be able to do the job to help the government to make sure these basic services are there."
The government describes the camps, which are encircled by barbed wire fences an dpoliced by soldiers, as "welfare villages", but Tamil activists have likened them to concentration camps.
But Ban said he did not believe that the security was there to stop the Tamils from leaving.
"They're trying to resettle them, to reintegrate them," he said. "That is the Sri Lanka government's commitment."
Military officials have previously said that the troops are needed because Tamil Tiger fighters could be hiding among the civilians.
Rajapakse told a religious gathering, held before his planned meeting with Ban, that he wanted ethnic unity and would work for reconciliation.
"We have a bigger responsibility now to rebuild our fractured nation," the president said.
Rajapakse declared the war with the Tamil Tigers, who were fighting for an independent homeland in the north and east of the island, over after government forces captured the last area held by the group and killed its leadership.
The Tamil minority makes up 12.6 per cent of the population of 20 million.
Vino Kanapathipillai, a Tamil activist and editor of the Tamil Guardian newspaper, told Al Jazeera that she was not optimistic that the Sri Lankan government would listen to Ban's calls.
"It has been our experience over the last five months that numerous international calls on Sri Lanka, for example to not shell civilians or take civilians into consideration as it pursued the war, have been ignored," she said.
"The Sri Lankan government has expressed its views towards the Tamil people quite clearly by its refusal to stop its shelling, by its willingness to progress the war at any cost.
"Until the conditions that created genocide and thus led to the Tamil military resistance are changed, there is going to be very little chance for unity."
The UN estimates that up to 100,000 people have died during the conflict, including at least 7,000 civilians killed since the beginning of the year.