'Sick as parrots'
Tuesday's budget was the first for Britain's coalition government, comprising the Conservative and Liberal Democrat parties, since it took power following elections in May.
Referring to the previous Labour government, Osborne said: "The years of debt and spending made this [budget] unavoidable."
However, the minister also announced plans to help the lowest paid by raising personal tax allowances to £7,475 ($11,000), taking about 900,000 people out of the tax system and giving millions of basic rate taxpayers a tax cut of $300 per year.
Speaking to Al Jazeera, Diane Abbott, a Labour MP running for the leadership of the party, said: "The problem with trying to get 80 per cent of the money out of public-sector cuts [is] it hits the poor and it hits middle income people.
"The Lid Dems bottled it, the Lib Dems were promising that they would take anyone earning up to £10,000 ($15,000), out of taxation, they bottled it, it has now gone up to £7,475 ($11,000).
"At the end of Osborne's speech, the Lib Dems looked as sick as parrots, none of them cheered, they all looked glum, because they knew they had failed to deliver."
'Tough but fair'
Describing the budget as "progressive" and "tough but fair," Osborne also announced cuts to housing benefits and family tax credits.
"Everyone will pay something but the people at the bottom of the income scale will pay proportionately less than those at the top," he said.
The new government is seeking to save tens of billions of dollars with state borrowing forecast to reach $230bn, or 10.5 per cent of gross domestic product (GDP), in the year to March 2011.
Osborne said the country's structural deficit should be eliminated in five years following the measures.
"We are on track to have debt falling and a balanced structural current budget by the end of this parliament" in 2014-15, said the minister.
Harriet Harman, the acting leader of Labour, said the budget was "reckless" and would lead to thousands of people losing their jobs.
She said the rise in VAT was "unfair" and went against pre-election statements by the Conservatives and Liberal Democrats, that they had no plans to increase it.
The UK's public deficit rocketed to a record-high in the last fiscal year, as a severe recession hit tax revenues and the government spent billions of dollars bailing out the country's banks.
The economy is predicted to grow by 2.6 per cent in 2011, according to the Office for Budgetary Responsibility, an independent fiscal watchdog set up by the new government.
That compared with a 3.25-per cent expansion forecast by the previous Labour government.