Borisov, nicknamed Batman after the fictional superhero due to his zeal for action, reiterated his campaign promises to move fast on changes.
"Those who have stolen should be very afraid ... The thieves will go to jail," Borisov, a former bodyguard, said on Monday.
A new government will have to quickly tackle the judiciary and economic policy to avoid new EU sanctions on aid, badly needed to fund Bulgaria's cash-strapped economy and to attract investors, many of whom fled this year.
This is likely to include starting loan talks with the International Monetary Fund and slashing government spending.
Borisov reassureed investors that "updating the budget was the first thing" his government would do, addressing concerns that Bulgaria might endanger its currency board regime by turning to a public deficit after years of surpluses.
"Those who have stolen should be very afraid ... The thieves will go to jail"
GERB party leader
GERB also won 26 additional seats in the 240-member legislature in a vote for a total of 31 mandates distributed separately, results showed.
Exit polls showed GERB might get a total of about 120 seats, one seat short of clear majority.
The former communist nation of 7.6 million people, which joined the European Union in 2007 and is the bloc's poorest member, lost access to over half a billion euros ($700 million) in EU funds last year as punishment for graft.
GERB's better-than-expected election result, which topped opinion surveys by 10 per centage points, alleviates concerns that protracted coalition talks could
water down reform plans.
Observers said GERB was likely to strike a deal with a grouping of rightist parties, the Blue Coalition, which was poised to take about 16 seats.
Some said Borisov lacks a track record of government work and his programme needs concrete policy plans, but they say he appears to have the political will
After 12 years of growth, Bulgaria is in recession and rising unemployment is ending years of voracious private spending that has fuelled debt.
The economy is seen shrinking by two per cent in 2009 and foreign direct investment vital for the economy is expected to halve this year.
Months of pre-election spending by the socialists also threatened to create a budget deficit that could deplete the country's hard currency reserves and undermine the lev's peg to the euro.
Five suspected criminals who stood in the election to escape prosecution failed to win seats in the next parliament, ballot results showed.
Loopholes in the laws of the European Union newcomer, allowed suspects awaiting trial to register as candidates in the election to secure release from custody and temporary immunity from prosecution.
If elected, their immunity would have been permanent.