A long-running dispute over Nagorno-Karabakh - an Armenian-majority enclave which broke free from Turkish-backed Azerbaijan after a war - has been a stumbling block towards reconciliation between the two countries.
Turkey closed its border with Armenia in 1993 to support Azerbaijan.
Saturday's accord, mediated by Switzerland, were signed in Zurich by Ahmet Davutoglu, Turkey's foreign minister, and Edward Nalbandian, his Armenian counterpart.
Last-minute disagreements had delayed the ceremony for three hours.
The accord is the culmination of more than a year of Swiss-mediated talks.
The protocols signed between the two countries would still need ratification by their respective parliaments.
That endorsement will have to come as nationalists on both sides protest the accord, particularly an Armenian diaspora which is demanding that Turkey acknowledge the killings of 1.5 million Armenians during World War I as genocide.
Turkey has disputed the claims of genocide, with support from the US and UK, saying that the real death toll is lower.
Many Turks see the fighting as a civil war caused by the collapse of the Ottoman Empire during which an unverifiable number of Turks also died - although both sides agree that more Armenians than Turks were killed.
Both governments have majorities in parliament but are expected to hold back on immediately ratifying the protocols due to the opposition.
"Al Jazeera's Anita McNaught, reporting from Karakoyunlu on Turkey's border with Armenia, said: "These protocols are powerful, but they have no legally enforceable status.
"They are reliant on the goodwill and moral authority of the parties who are the participants in it.
"What happens next is ratification ... they've got to sell this to their people and the politicians.
"If they push it though and they ratify it in parliament, we see two things: immediately, the establishment of diplomatic relations; then within two months the opening of borders."
For now, however, the question of reconciliation remains contentious at the very least.
As many as 10,000 people marched from Yerevan, the Armenian capital, to a hilltop memorial to World War I-era massacres on Friday to condemn the accords, some carrying placards reading "No Concessions to the Turks".
The move is expected to help Ankara in its bid to join the EU, while Armenia may benefit from closer ties to the West and greater economic openness with Turkey.