A Saudi newspaper, Okaz, on Sunday quoted the kingdom's security officials as saying the smugglers fled after border guards intercepted them on Friday in a mountainous region of Jizan province. They left 10 "high explosive" bombs and 50 rounds of ammunition, the paper said.
Saudi Arabia has seized tons of explosives and ammunition in the last year and tightened controls over its desert and mountain borders.
But its southern neighbour Yemen complained a barrier along their joint border violated a treaty establishing a demilitarised zone on both sides of the boundary.
Yemeni Foreign Minister Abu Bakr al-Qirbi said after a two-day summit last week Saudi Arabia had agreed to dismantle the barrier, which he said stretched for tens of kilometres along the 1300km border.
Yemen says the barrier is in parts a raised concrete-filled pipeline and in others a sand barrier.
Saudi Deputy Interior Minister Prince Ahmad bin Abd al-Aziz said after the talks his country had built "hurdles such as sand dikes" on its side of the border and was discussing construction of similar dikes on the Yemeni side.
Saudi security forces have frequently clashed with smugglers from Yemen - the governor of Jizan said last year border guards "find an arms haul every hour" - but the dispute over the barrier has put the problem in the spotlight.
Saudi officials are likely to point to Friday's night-time arms seizure as proof that its unsealed borders continue to give smugglers the chance to supply those behind last year's bombing that killed more than 50 people in Riyadh.
A joint statement issued after the Saudi-Yemeni talks said the two sides agreed to jointly patrol the border and set up observation stations to curb smuggling.