"We anticipate that the remaining seven will be released this afternoon," presidential spokesman Chon Ho-seon said at a news briefing.
Earlier Greg Muller, an ICRC representative, confirmed that 12 hostages had been released on Wednesday and taken to the Red Crescent Society offices in Ghazni, 140km south of Kabul.
"They seem, after six weeks in detention, very much relieved which is a natural reaction after an extremely stressful experience," Muller said.
The South Korean embassy in Kabul said the freed hostages were likely to be flown to the US military base at Bagram, north of Kabul, before leaving Afghanistan "as soon as possible".
Earlier the Taliban released eight hostages, saying that more would be released soon.
The Taliban began freeing the South Koreans, a group of Christian volunteers who had travelled to Afghanistan to undertake aid work, as part of an agreement reached a day earlier in face-to-face talks with a delegation from South Korea.
Under the terms of the deal, South Korea agreed to end missionary activities by Christian groups in Afghanistan.
But there has been speculation that the South Koreans bought the release of the hostages, though both the Taliban and the South Korean government denied there was any secret deal.
Alan Fisher, reporting for Al Jazeera from Kabul, said: "We've certainly heard rumours around Kabul - the figure of around 20 million pounds has been bandied around.
"I spoke to one senior Afghan authority who, while not confirming the figure, did say that money was paid - that the South Koreans had paid cash to the Taliban."
But Qari Mohammad Bashir, a Taliban commander, denied that a ransom had been paid.
"I strongly deny this. It's not true that money was involved," he said.
Al Jazeera's Tony Birtley, reporting from South Korea, said: "Most people here [in Seoul] think that South Korea has probably paid a ransom, but that will be debated later when the hostages have returned home safely."
|Relatives reacted with joy after getting news|
that an agreement had been reached [AFP]
Relatives of the hostages, who erupted in cheers on hearing news of the agreement, are now anxiously awaiting the hostages' return.
"It is like a dead child is coming back to life," Lee Hyoen-Ja, a relative of one of the kidnapped Christian aid workers, told JoongAng Daily on Wedneday.
Seo Jeung-Bae, whose son and daughter were among the hostages, said: "I want to see them and hug them hard now.
"I had not doubted for one moment that the Taliban would return my children some day, as the Taliban are also human beings and have their own families."
At Seoul's Saemmul Church, which sent the volunteers to Afghanistan, officials said the focus now would be on looking after the released hostages and their families.
"Our work for now will be to make sure the freed hostages return safely and have the time to recover, and to make sure the family members of the two who were sacrificed are comforted," Bang Yong-kyun, pastor, said.
The group of 23 volunteers from the church were seized on July 19 from a bus as they travelled through Ghazni province.
The Taliban killed two male hostages early on in the crisis, but released two women as a gesture of goodwill during a first round of negotiations.
As news of the release spread, other South Korean churches said the kidnapping crisis had led them to rethink evangelical missions to Afghanistan.
The National Council of Churches in Korea, one of the largest groups representing the country's Protestants, said in a statement it would abide by the government's pledge to end missionary work in Afghanistan.
"Through this incident, we will look back on the Korean churches' overseas volunteer and missionary work, and make this an opportunity to bring about more effective and safer volunteer and missionary work," it said.
Another Seoul-based Christian aid group, The Frontiers, said all its short-term volunteers in Afghanistan had pulled out of the country and two long-term volunteers are about to return.
Following Tuesday's talks with South Korean officials, the Taliban said they would release the 19 hostages provided Seoul pulls out its troops and stops Korean missionary work in Afghanistan by the end of this year.