The discussions were part of efforts by the two countries to work toward establishing diplomatic relations under an agreement in six-party talks last month by North Korea to start shutting down its nuclear weapons programme in exchange for millions of dollars in energy aid.
It is not clear if the negotiations will continue at a later date or if the discussions are to be postponed indefinitely.
Nuclear negotiations in doubt
Japanese and North Korean officials did not say how - or if - the breakdown in talks would affect ongoing international attempts to persuade North Korea to dismantle its nuclear programme in return for diplomatic normalisation and an increase in aid.
Before the talks, Tokyo's chief envoy Koichi Haraguchi said North Korea must settle all lingering questions about its past abductions of Japanese citizens before the two nations can draw closer.
The meeting is a crucial step in North Korea's agreement last month to shut down its nuclear reactor in exchange for aid and other concessions, Haraguchi said.
"Normalisation is impossible unless the abduction issue is resolved," Haraguchi told reporters before the talks.
Calls for reconciliation
North Korea admitted in 2002 that its agents abducted 13 Japanese, sparking outrage in Japan. Five of those were repatriated and Pyongyang said the other eight are dead.
Earlier on Wednesday, Koichi Haraguchi, Japanese head negotiator said: "Our biggest purpose is to urge them to deal with this issue squarely and take positive measures."
"Our biggest goal is early return of all the abductees."
For its part, North Korea was expected to press for settlement of issues stemming from Japan's harsh 1910 to 1945 occupation of the Korean peninsula.
"I hope we, recognising the sense of responsibility and commitments, will help with the implementation of the six-party agreement by proceeding with our working-level talks," Song Il-ho, North Korea's negotiator, said in his opening statement.
"We believe it is favourable for us to build substantial political, economic and cultural relations by settling the unfortunate past and issues of mutual concern."
The talks were mandated by a February agreement between North and South Korea, China, the United States and Russia that aims to eliminate the North's nuclear weapons program.