Secularist Turks and the main opposition Republican People's Party (CHP) remain hostile to Gul's bid, arguing that his presidency would undermine the country's secular system.
Gul was initially expected to take the presidency in the first-round vote, after the AKP gained seats in July's general election.
The election was called early to defuse a crisis over the presidency.
However, with other parties fielding their own candidates, votes for Gul just fell short of the amount needed to win.
The CHP was absent during the first-round vote, signalling its continued opposition to Gul's attempt to become president.
Prior to the vote, the Kurdish Democratic Society Party (DTP) also said it would not vote for Gul.
Ali Riza Alaboyun, an AKP member, told Al Jazeera that while opposition remained to Gul's candidacy, the foreign minister would be a worthwhile president.
"I think prejudice is prevailing in this situation. [The opposition] will see that Mr Gul will be one of the best presidents. He will not be biased. He is a real gentleman. The world knows him and he has real experience," he said.
"Our constitution clearly defines who will be president. If a politician becomes president, the constitution says he must resign from his political party."
Up to four rounds of voting can be held to elect a new president for Turkey.
Despite his failure to secure enough votes in Monday's vote, Gul remains likely to be elected in the third session, on August 28.
Gul will only require a simple majority to win in that vote, which the AKP has.
The opposition National Movement party (MHP) also put up a candidate, Sabahattin Cakmakoglu, for Monday's vote.
The Democratic Left Party (DSP) fielded Tayfun Icli as their candidate.
The first vote in April was derailed by a court ruling that two-thirds of parliament had to be present - impossible amid an opposition boycott.
The secularist army, which undermined the April vote with a sternly worded anti-government statement, has signaled it has said all it plans to say about Gul's ambitions.
As recently as 1997, the army removed a government in which Gul served over a perceived programme of political Islam.
Gul has said he supports secularism but opposition from secularists remains fierce, in part because his wife wears a headscarf.
Some critics fear he wants to break down the division between state and religion.