"The referees have been very tough, in my opinion over tough," said Ferdinand.
"A lot of tackles that have been booked, well the yellow cards have been handed out too freely.
"But that's the way it goes and we've got to deal with that and the team that deals with that best will be the team that wins."
Bayern Munich player Hargreaves respected the referees involved at the tournament but asked for more understanding from the whsitleblowers.
"All the referees, just like the players, you've got the best refs in the world," the current England right back said.
"But I think some decisions have been hard for some of the players to understand because they have been blowing the whistle so much.
"Obviously it's important for the players that the ref has a good day and lets the players play because it is very difficult to get a flow to the game if it's stopped every 30 seconds."
Portugal’s last match against the Netherlands saw a World Cup record 16 yellow and 4 red cards being handed out.
Referees have come under criticism from a number of quarters.
Bernard Saules, president of the French National Union of Referees, deemed the officiating 'catastrophic'.
"They have selected the best teams from each continent, but not the best referees," said Saules.
"It has been a catastrophe from the start."
World Cup organiser Franz Beckenbauer claimed that referees were failing to keep a grip on matches.