The cause of Monday's crash in an urban area near Amagasaki, 410km west of Tokyo, was not immediately known.
But Aljazeera's correspondent in Tokyo, Faid Salama, said human error on the part of the inexperienced, 23-year old driver might have caused the accident, considered the worst in the last 42 years.
The survivors, too, said excessive speed might have been a factor.
Search and rescue operations were continuing throughout the night, with floodlights and soldiers pressed into action.
The driver is believed to have overshot the stop line at the last station before the accident, Salama added.
The train operator, West Japan Railway Co, apologised: "Our most important task now is to rescue the passengers from the accident and we are doing our best," company President Takeshi Kakiuchi said.
Survivors said the force of the derailment sent passengers tumbling through the inside of the cars.
Photos taken by the national broadcaster (NHK) aboard the train showed passengers piled on the floor and some clawing to escape from the busted shells of the cars.
Officials said there were 580 passengers aboard the train.
NHK said the train collided with a car in between train stations while it was running at a speed of 70km per hour.
The train may have been
going at 133km an hour
Investigators struggled to come up with reasons for the crash. Tsunemi Murakami, the train operator's safety director, estimated that the train would have been going 133km an hour to have jumped the track purely because of excessive speed.
He said it still was not certain how fast the train was running at the time of the accident.
The front car of the train smashed into the apartment building and the second and third cars were derailed in the accident, which occurred shortly after the morning rush hour on Monday.
Television showed rescuers with stretchers trying to remove people from the wreckage. Witnesses said there were not enough stretchers, so rescuers were stripping train seats to carry out the injured.
"There are a lot of seriously injured people, some with fractures," passenger Tatsuya Akashi, who was on his way to work, said.
"I've got a lot of blood on my clothes. I don't know what happened."
He added: "It felt like the train speeded up as it was going around a curve and I thought there were some strange swings, and then the train derailed."
"No one knew what happened, and everyone kept screaming."
''No one knew what happened, and everyone kept screaming''
A passenger on the train
Aljazeera's correspondent said Japanese trains generally had a good safety record.
While the trains that run outside the cities are equipped with high-tech equipment, the ones that run within them are not.
However, even the latter are equipped with an automatic-control systems and but for this equipment, Monday's crash toll would have been much higher.
In the last major accident, in March 2000, five people were killed and 33 were hurt when a Tokyo subway train ripped away the side of a carriage of an oncoming train that had derailed in its path during rush hour.
The worst Japanese train crash in recent history took place in 1991 in Shigaraki, western Japan, when 42 people were killed and more than 600 injured.