An army spokesman said the accident occurred on Saturday while the soldiers were hanging from a helicopter on ropes, which the chopper-crew were forced to off when a sudden gust of wind threatened to bring down the aircraft.
"They fell into the sea about 300 metres from the shore," Lieutenant Colonel Achmad Yani Basuki said.
"We sent a fast motor boat to hunt for them but when they arrived in the area they could not see them and we fear they may have drowned," he said.
He declined to say which unit the men were from but the Antara state news agnecy said they were from the Special Forces Command, known as Kopassus.
The soldiers were practicing for the Armed Forces Day on Sunday.
On the northern tip of Sumatra island, Aceh has been under martial law since 19 May after the Indonesian army launched a massive offensive against separatist rebels.
At least 839 suspected rebels, 52 Indonesian soldiers, 11 policemen and 300 civilians have been killed in the continuing conflict.