He said: "The process may create strains in some circles of Somali society but peace should prevail between brothers, depending on the wisdom and patriotism of Somalis both inside and outside the country."
He also said the talks will be "without external influence".
The closed-door discussions are expected to last up to a week.
Ould Abdallah has also written a letter seeking the support of the Somali people in the diaspora to back the ongoing peace process.
"I am asking for your continued support for the ongoing process and our challenging undertaking to return Somalia to a normal, stable country back in the international arena," he wrote on Wednesday.
Since taking office in November, Hussein has engaged with Somalia's mainly Islamist opposition - unlike Ali Mohamed Gedi, his predecessor.
Abdirahman Osman, a political analyst and founder of the Somali Concern Group, told Al Jazeera that the situation inside the country has prompted some opposition figures to hold talks, despite them being against the presence of Ethiopian troops in Somalia.
He said: "That is why Sheikh Sharif Sheikh Ahmed [leader of the main Somali Islamist alliance] decided to attend the talks, the humanitarian situation has become so terrible."
"However, the transitional federal government do not want Ethiopian forces to leave, unless they are replaced by a UN contingent of peacekeepers."
Somalia has been wracked by conflict since 1991, with Mogadishu, the capital, plagued with political and civil unrest, food riots, and attacks on Western aid agencies.
A civil war that has lasted for nearly two decades has defied more than a dozen peace initiatives since Mohamed Siad Barre, the former president, was removed from power in 1991.
The Union of Islamic Courts (UIC), a militia which ousted the government from Mogadishu in 2006, briefly ruled large parts of the country before being defeated last year by Ethiopian-backed Somali government troops.
Government troops are still battling the al-Shabab, the UIC's military wing and allied clans in a protracted conflict which has left thousands of people dead and hundreds of thousands displaced.