The MDC leader has selected 14 cabinet ministers but Mugabe, who was supposed to have 15 ministers sworn in, instead brought 22 as the swearing-in ceremony got underway.
The MDC splinter group, MDC Mutambara, has three cabinet posts.
But in the latest of a series of political twists, Roy Bennett, a white farmer who became treasurer of the MDC, was arrested at a Harare airport just an hour before the oath-taking ceremony, MDC sources said.
Bennett returned just last month from three years of self-imposed exile in South Africa, where he had fled to escape charges of plotting to kill Mugabe, and was set to become deputy agriculture minister in the new government.
Al Jazeera's Haru Mutasa, reporting from Harare, said: "He [Bennett] is still in police custody some where outside the capital. People are confused as to what is going on."
She said some people were wondering why Tsvangirai had agreed for his ministers to be sworn in in the first place when one of them was being detained.
"They are trying to locate Mr Bennett at the moment. They are really not sure where he is. Once they cross that stage then the MDC will maybe give more clarity on what their position is."
Mutasa said there was "nothing new" in the cabinet list as Mugabe had sworn in staunch supporters who have served in his government for decades.
"It's still the old guard and if they come back with the same ideals, the same way of thinking, it will be very difficult for the new prime minister to actually work with them," she said.
"How will he put two different theories, two different groups of people together, how will they come out with one united front to help Zimbabwe to actually find a solution to the humanitarian crisis they are facing?
"Tsvangirai has a lot of work to do, a short time to do it in and people are saying it has to happen quickly or there is a danger that some of his supporters might lose patience with him."
Isabella Matambanadzo, Zimbabwe's programme director for the Open Society Initiative for Southern Africa, said: "It's really critical that this government comes together and delivers together.
"We want to know that we can live in a Zimbabwe where we will not be tortured by our own security forces. We also want to live in a Zimbabwe where we have predictability in our economy."
Control of the home affairs ministry, which oversees Zimbabwe's police forces, was one of the sticking points in power-sharing talks between Mugabe and Tsvangirai.
Zanu-PF and the MDC will co-manage the ministry in the new government.
'Willing to fight'
The crucial ministries of defence, justice and foreign affairs have remained under the control of Mugabe, who has been Zimbabwe's president since 1980.
Tsvangirai has selected Tendai Biti to the post of finance minister in an attempt to control soaring inflation which has rendered the Zimbabwe dollar practically worthless.
|A cholera epidemic is one of the most urgent challenges the unity government faces [AFP]
Biti, who has in the past been accused by Zanu-PF of treason, said that political unity depends on the actions of Mugabe's party.
"There has to be mutual respect - we are not going into [the cabinet] to fight. If we do have to fight, we will give as much as we get. But that is not the spirit that I want to see," Biti told Al Jazeera.
"The spirit is that we are going to put Zimbabwe first."
Massive foreign investment is required to help rebuild the country, but western countries say that such investment depends on whether there is long-term political stability.
"Until the government of Zimbabwe could convince us that there were going to be free and fair elections, and at the same time that there was going to be the removal of repressive legislation ... until these things happened, we could not treat Zimbabwe as if it was an ordinary country," Gordon Brown, Britain's prime minister, said.