Armenians at the service carried placards such as "Nobody and nothing will be forgotten!" and "Genocide never gets old".
In an address to the nation on Saturday, Serge Sarkisian, the Armenian president, described the killings as "unprecedented in its scope, monstrosity and graveness of its consequences".
"We are grateful to all those in many countries, including Turkey, who understand the importance of averting crimes against humanity," he said.
Al Jazeera's correspondent Anita McNaught was in Istanbul where people had turned up to commemorate the event that Armenians around the world call "Genocide Day".
"It is an unusual event, this event would not have happened two years ago in Turkey at all, it is a measure of the degree to which the ability to discuss the issues here in Turkey has freed up," she said.
"People gathered in the square not to protest or engage in an act that might be seen as aggressive, they have come here as Turks and Armenians, and relatives of Armenians living in Turkey sitting side by side.
"They are all here to commemorate the pain that all of Turkey shares. There are no protest banners; the slogans being used say, 'This pain is our pain, this mourning belongs to us all'."
Several historians have described the event as the first genocide of the 20th century.
The slaying began on April 24, 1915, they say, with the rounding up and murder of about 800 Armenian intellectuals.
The Ottoman authorities then evicted Armenians from their homes in actions that the historians say expanded into the mass slaughter of the Armenian population.
Scholars sympathetic to the Turkish interpretation of the events reject the use of the word "genocide" to describe the kilings.
Debate over what to call the massacre has disrupted relations between Armenia and Turkey for decades.
Turkey has warned the US administration of diplomatic consequences if it fails to prevent the passage of a congressional resolution that would recognise the "killings of Armenians genocide".
The foreign affairs committee of the US House of Representatives last month passed a resolution declaring the killings "genocide", but it is unclear whether the chamber will vote on it when it is passed on.