Three infants and 11 crew were among those on board.
The plane, carrying mostly French and Comoran nationals, was flying from the Yemeni capital, Sanaa, to Moroni on the main Comoros island of Grande Comore.
Most passengers had travelled to Sanaa from Paris or Marseille on another aircraft.
Search and rescue planes had spotted debris at the crash site, said Ibrahim Kassim, an official with the Agency for Aviation Security and Navigation in Africa and Madagascar (Ascena), which covers Francophone Africa.
He said the plane had probably come down between five and 10km from the coast and crashed along its landing approach.
"The weather is really not very favourable. The sea is very rough," Kassim told the Reuters news agency.
Moroni airport authorities said civilian and military rescue teams were immediately deployed to search for the plane in the rough waters.
"Two French military aircraft have left from the islands of Mayotte and Reunion to search the identified zone, and a French vessel has left Mayotte," Hadji Madi Ali, the director-general of the airport, said.
Mohammad al-Sumairi, the deputy general manager for Yemenia operations, told Reuters that they did not know the cause of the crash.
"The weather conditions were rough with strong winds and high seas. The wind speed recorded on land at the airport was 61km an hour," he said, adding there could be "other factors".
Comoros officials said the plane could have crashed in the area of Mitsamiouli, a town on the main island.
The Comoros covers three small volcanic islands, Grande Comore, Anjouan and Moheli, in the Mozambique channel about 300km northwest of Madagascar and about the same distance east of the African mainland.
The crash marks the second time an Airbus has plunged into the sea this month, after an Air France Airbus A330-200 crashed into the Atlantic Ocean killing 228 people on board on June 1.