Taro Kono, the vice-justice minister and head of the panel, suggested limiting the proportion to 3%, compared with 1.2% now, a ministry spokesman said on Wednesday.
He said: "Some countries accept five or ten per cent, but that would be absolutely impossible for Japan."
The public fears that increased immigration would lead to a rise in crime.
A press release on the report said that people with specialist skills should be favoured over manual labourers.
The panel also proposed tightening immigration requirements on foreigners of Japanese descent who can work in Japan with few restrictions.
“We should not accept people simply on the basis that they are of Japanese descent,” the document said.
About 250,000 foreigners, mainly Brazilian, were living in Japan on such terms by the end of 2004. The document said that those already in the country should have to learn basic Japanese and be able to earn a living to be allowed to remain.
The proposals come at a time when many countries are looking to curb immigration. The United States, Germany, Belgium, Portugal, Italy, Greece, Holland and Britain have all enacted legislation related to immigration.
Jose Luis Rodriguez Zapatero, Spain's prime minister, announced a plan to control immigration on Tuesday.
Nicolas Sarkozy, the French interior minister, submitted a bill to parliament this month to admit only skilled immigrants.