The Swiss army said it was possible the victims were recruits on a training exercise.
"We fear that recruits from mountaineering units were involved," Daniel Laroche, army spokesman, said.
He said a patrol of six men was in the area at the time of the accident, but added that he was unable to provide confirmation at this stage.
Summer is the peak climbing season in the Swiss Alps, attracting climbers from all over the world.
The 4,158m Jungfrau and the nearby Eiger are favourite destinations and regarded as tough climbing challenges.