The accord was signed by Saudi Prince Talal bin Abd al-Aziz, a half-brother of King Fahd who heads the Arab Gulf Programme for UN Development Organisations (AGFUND), and Yemeni Labour and Social Affairs Minister Abd al-Karim al-Arhabi, the official SPA news agency reported.
Bank of Hope, as it has been dubbed, will be funded by AGFUND and contributions from Arab countries.
In October 2000, Yemen signed an agreement with Riyadh-based AGFUND to set up the bank with capital of $5.5mn.
AGFUND said at the time it would provide up to $2mn capital to the bank, with the Yemeni government putting up $2.5mn and a further $1mn coming from the private sector.
"This banking system will allow the poor increasingly to become partners of the bank and shareholders," Prince Talal said, adding that Egypt, Lebanon and Jordan had agreed to set up similar banks.
Founded in 1981 at Prince Talal's initiative, AGFUND provides grants for UN projects and coordinates aid from the oil-rich Gulf Arab states.