A South Korean defence ministry official said the boat's satellite navigation system appeared to have malfunctioned, causing it to stray into North Korean waters.
"It is not likely a case of a North Korean vessel venturing into the South and capturing the ship," the official said.
Seoul sent a phone and fax message to Pyongyang through an inter-Korean maritime communication line to seek the crew and boat's release, with Pyongyang responding later through the same communication line, Yonhap reported.
The incident comes amid rising tensions on the Korean peninsula, with Pyongyang conducting its second nuclear test, test firing a barrage of rockets and issuing bellicose rhetoric towards the South in the last few weeks.
On Sunday, North Korea threatened the South and the US with "unimaginably deadly blows" should it come under attack.
|A South Korea official said the boat's equipment probably failed [EPA file]
"We will mercilessly and resolutely counter the US imperialists and the South Korean puppets if they ignite a war, obsessed with a foolish ambition," Kim Yong-chun, North Korea's defence minister, was quoted by the official Korean Central news agency as saying.
The two Koreas are technically still at war having signed just an armistice in 1953 to end the Korean war and even that has been abandoned by Pyongyang in recent months.
The capturing of the fishing boat may raise fears for the freedom of the four people on board.
Along with disagreements over rent and wages at a jointly-run industrial park on the northern side of the border, the North has held a South Korean worker from there for more than 120 days for allegedly insulting the North's political system.
Pyongyang also continues to hold two American journalists who were captured at its border with China in March and found guilty of "hostile acts" against the North.