Detlev Mehlis, 55, senior public prosecutor in the office of the attorney-general in Berlin, was named by UN Secretary-General Kofi Annan to head the international inquiry earlier this month.
"I'm very glad to be here even if it is because of an incomprehensible tragedy, not only for you but for everyone in the civilised world," Mehlis said after arriving on Thursday.
Al-Hariri's assassination on 14 February led to massive demonstrations in Lebanon and calls for Syrian troops to leave the country, which they did last month.
The German prosecutor has been given a three-month renewable deadline to come to a conclusion.
Investigators have already submitted a report casting suspicions on the roles of Syria and the Lebanese security services in connection with al-Hariri's death.
Al-Hariri's son Saad, a candidate in Lebanon's forthcoming parliamentary election, said earlier this month he expects "many people in Lebanon to ask for political asylum with the arrival of an international investigator".