Some financial analysts fear the case could threaten the bank secrecy laws that underpin Switzerland's wealth management industry.
Gruebel, a 66-year-old German and a four-decade veteran of the banking industry who started his career at Deutsche Bank, immediately warned UBS staff to brace themselves for more job cuts.
In a statement, he said: "The opportunity to lead UBS with its unique client franchise in wealth management, investment banking and asset management in these extraordinary times presents a fascinating, yet formidable challenge to me."
In an internal memo to staff, a copy of which was obtained by Reuters, Gruebel said "further substantial cost reductions" were inevitable given the tough business climate.
UBS had already announced another 2,000 investment banking job cuts this month to shrink its total workforce to around 75,000 by mid-2009 from a peak of about 85,000.
It had said it was starting to turn the corner, with net inflows into both its wealth and asset management units in January.
However, news of the widening US investigation pushed its shares down to new all-time lows this week, piling more pressure on top executives to step down.
Gruebel performed a rescue act at Credit Suisse, where as co-CEO and then CEO from 2003 to 2007 he reversed multi-billion-dollar losses and led a sweeping restructuring that restored profitability and confidence in the group.
He said Switzerland needed more than one big bank after talk that UBS might be forced to merge with Credit Suisse, and said he would work hard to return the bank to profit.
Dirk Becker, an analyst at Kepler Capital Markets, a European financial services company, said: "Although Rohner was only appointed after the problems started and he can hardly be blamed, it was under his leadership that UBS had six consecutive loss-making quarters."
"With Gruebel as the new CEO the bank can now restore confidence with investors, clients and regulators."
Peter Thorne, an analyst at Helvea, a Swiss broker, said: "Rohner had failed to implement the bank's strategy outlined in the summer of 2008 to make wealth management the core operation, and the bank was drifting along without any clear sense of direction.
"Gruebel has the strength of character and banking knowledge to take charge at UBS and actually implement a strategy."
Doris Leuthard, the Swiss economy minister, said the change came at the right time.
"I now hope that the trust in the new UBS returns quickly," she said.
The bank's shares rose 10.1 per cent in early trading on Thursday.