Bardasar Akhpar, a demonstrator, said: "We are here because we want to tell the entire world that we do not forget the genocide of 1915.
"We will not welcome Gul nor any other Turk until they have recognised the genocide."
Gul was taken to meet Serzh Sakisian, the Armenian president, after being invited by him to attend a world Cup football qualifier between Armenia and Turkey at Yerevan's Hrazdan stadium.
The invitation was extended despite the fact the two countries do not share diplomatic relations.
On meeting Sarkisian, Gul offered the Armenian leader the opportunity to watch a return football match between the two countries in Turkey next month.
"I hope that this visit will create the possiblity to improve bilateral relations," said Gul at a joint press conference with Sarkisian in Yerevan.
Sarkisian said the visit there is a "political will to decide the questions between our countries, so that these problems are not passed on to the next generation".
Armenians say that up to 1.5 million of their people were slaughtered by Ottoman Turks as their empire fell apart at the height of the First World War.
Yerevan's claim has won support from several other countries.
Turkey rejects the accusation and says that 300,000-500,000 Armenians and at least as many Turks died in civil strife after Armenians took up arms for independence in eastern Anatolia.
Nadim Baba, Al Jazeera's correspondent in Yerevan, said Armenians appear to be holding out for improved relations with their country's westward neighbour.
"From the people that we have spoken to on the streets of Yerevan, I would say that the majority are longing for better relations with Turkey, while being very much concerned that their government do not give away too many concessions to Ankara," he said.
"They do not want to let go of the hope that one day the world will recognise what happened almost a hundred years ago as a genocide.
"They also want to see their economy improve through better relations with Turkey and other countries in the region."
Ali Babacan, Turkey’s foreign minister, said diplomatic ties between Ankara and Yerevan would be discussed between during talks between Gul and Sarkisian but he a major breakthrough was unlikely.
"I do not think we should raise expectations that high … But on the other hand, when we open the doors for dialogue, that means we are ready to talk about the problems," Babacan said.
"It is my wish that this match will help lift the barriers dividing two people who share a common history and will contribute to regional friendship and peace," Gul said ahead of his visit.
Turkey has refused to establish diplomatic ties with Armenia since the former Soviet republic gained independence.
Turkey also shut its border with Armenia in 1993 in a show of solidarity with its close ally Azerbaijan, then at war with Armenia over Nagorny Karabakh, a secessionist Armenian-majority region in Azerbaijan.