No membership decision
Tuomioja said after meetings with the foreign ministers of Turkey and Cyprus: "Unfortunately ... at this stage circumstances do not permit that an agreement could be reached during the Finnish Presidency."
He also said that the EU would decide on how the failure to find a solution to the dispute would affect Turkey's EU membership bid.
The EU's general affairs council is set to meet to discuss the issue on December 11.
Speaking after the breakdown of the talks Ollie Rehn, the EU enlargement commissioner, said: "Negotiations will not be stopped or frozen, they will continue more slowly."
Turkey said the failure to break the deadlock in its row with Cyprus over trade should not harm its bid to join the bloc.
Murat Ozcelik, a spokesman for the Turkish foreign ministry, said: "The failure of the Finnish proposals, despite our constructive attitude, should not affect our negotiations negatively."
EU observers have said the EU could sanction Turkey by freezing some of the 35 accession chapters which all EU candidate nations must satisfactorily complete before being allowed into the club.
Cyprus has opposed this idea, saying it would have no effect as other chapters, on sectors unrelated to customs or trade, could be opened instead.