The ministry earlier on Monday said the couple had been freed, based on information obtained from the Yemeni authorities by Swiss diplomats in the region.
"The Swiss couple abducted in Yemen has been freed. Both are safe and well," the Swiss Foreign Ministry in Berne said in a statement on its website.
But Hassan Muhammad al-Dhaman, leader of tribesmen holding the pair, said he was still in possession of the Swiss couple and that demands that his imprisoned brother be released by Yemeni officials had not been met.
"My brother is still being held by authorities, and the tourists are still here. They are sleeping now," Dhaman told Reuters by telephone.
A Yemeni official said he had no information about the hostages being freed.
A Swiss Foreign Ministry spokesman was not immediately available for comment.
The couple were seized early on Monday morning.
"They remain in good health," the Yemeni Interior Ministry said. It did not release their names.
Sources in Yemen said they were kidnapped in Maarib, east of the capital.
One of the alleged kidnappers said the pair would be freed soon if the authorities hand over his arrested brother.
"My brother is 30km away and being brought over to us now by authorities. As soon as he is here, I will personally hand over the tourists," al-Dhaman said by telephone.
Al-Dhaman earlier cautioned that the pair would be killed if the government tried to free them by force.
"I hold the government responsible. If they launch an attack, we will all die together as we are in the same house," he said. "The two tourists are in good health and being treated well."
Tourists have often been kidnapped by tribal groups in Yemen, a poor country at the tip of the Arabian Peninsula. Most have been released unharmed after negotiations.