Negotiations in Morocco have been focused on replacing the ban with a closely monitored quota system for the three countries for 10 years.
"As we can see it today, we do not believe these negotiations will succeed," Karsten Klepsvik, the Norwegian Commissioner to the International Whaling Conference (IWC), said on Tuesday.
"There are at least eight [to] 10 stumbling blocks, but the main stumbling block is that those who are against whaling seem to be willing to accept nothing but nil [quotas], and we cannot accept that," he told reporters.
Glenn Inwood, the spokesman for the Japanese delegation at the IWC, also said in an interview that, "Japan has compromised ... anti-whaling countries have offered nothing".
"If this process is going to survive, it requires compromise from both sides," he added.
Japan is viewed by campaigners as key to the outcome of the meeting of the 88-member IWC.
The Asian nation has said that it is willing to give up opt-out clauses in the almost 60-year-old whaling convention, which have given Iceland and Norway the opportunity to continue whaling and be a member of the IWC.
It has additionally said it will allow international inspectors onto whaling vessels and supports the development of a DNA registry to facilitate inspections of whale meat products.
Inwood said that Japan had offered to halve its self-determined quota of whales hunted in the Southern Ocean.
He also praised efforts by other conservation nations, such as the US and New Zealand, to find a middle ground.
Pro-whaling nations have said that the proposed quota system, in which the number of whales permitted to be killed diminishes annually, is a ploy to outlaw the practice altogether after the system expires.
During the 2008-2009 whaling season, Japan, Norway and Iceland harvested more than 1,500 of the marine mammals.
Anti-whaling campaigners have said that the draft quota system is a sell-out and that there should be stricter enforcement of the current ban instead.
"Any proposal that comes onto the floor of the commission that seeks to remove in effect the moratorium on commercial whaling ... in our view should not be adopted nor voted on," Peter Garrett, Australia's environment minister, said.
Australia, Germany, Britain and most Latin American countries have taken a hard line against the whaling nations on hunting in the Southern Ocean, which was declared a whale sanctuary in 1994, and trade.
Iceland is seen as unwilling to renege on the trade of whale.
Negotiations will also concentrate on the size of quotas and the species of whale to which they will apply.
The ban came into existence after some whaling species came close to extinction.