The negotiations centred on compensation issues related to Japan's 1910-1945 colonial rule over the Korean peninsula, and Tokyo's demands for more information on North Korea's past abductions of Japanese citizens.
Despite the lack of a firm deal or agreement, both sides remained optimistic.
"We were unable to solve pending issues during the two-day talks, but it was meaningful that we could have thorough discussions," Japanese envoy Yoshiki Mine told reporters after the meetings.
"I think we made certain progress"
"I think we made certain progress," he said.
Kim Chol Ho, the head of the North Korean delegation, said it may be possible for the North to discuss Japan's demand for more investigations into the kidnappings.
"If the environment (for better relations) is created and a trusting relationship is established, I think this is an issue that can be discussed,'' Kim was quoted as saying by Japan's Kyodo News agency.
North Korea admitted in 2002 it had kidnapped 13 Japanese citizens in the 1970s and 1980s.
"The abduction problem is an important issue. But it is meaningless if we are just holding talks unless we make progress"
It allowed five to return to Japan with their families, but said the other eight were dead.
It has since insisted the issue is settled, while Japan has demanded proof of the deaths and says more of its citizens may have been taken.
On Thursday Shinzo Abe, the Japanese prime minister, expressed frustration over the issue.
"The abduction problem is an important issue. But it is meaningless if we are just holding talks unless we make progress,'' he told reporters in Tokyo.
Another sticking point has been Japanese demands that North Korea hand over several Japan Red Army Faction terrorists who led the 1970 hijacking of a Japan Airlines plane to Pyongyang.
They are suspected of helping North Korean agents abduct several Japanese citizens from Europe.
For its part North Korea has demanded reparations from Japan for its colonization of the Korean peninsula in the years before and during World War Two.
Japan has yet to formally apologise to North Korea for its wartime actions.