Yekhanurov won only 223 votes on Tuesday, three less than he needed to be named to the job.
President Viktor Yushchenko had tapped Yekhanurov to replace his Orange Revolution ally Yulia Tymoshenko, whom he fired on 8 September.
The defeat was a major blow for Yushchenko, who came to the parliament session hall to plead with lawmakers to accept his choice, warning that a "no" vote would support those trying to destabilise the republic.
But Yushchenko failed to win the support of Ukraine's opposition parties and those factions now loyal to Tymoshenko, whom many Ukrainians see as the heroine of last
year's mass protests.
Officials have said that if Yekhanurov failed to win enough votes they would persist with negotiations.
Meanwhile, Kiev's leading prosecutor has announced that investigators have opened five criminal cases against officials from the country's top security body.
Many see Tymoshenko as the
Orange Revolution's heroine
"The criminal cases are open, the investigation continues," said Prosecutor-General Svyatoslav Piskun.
The announcement followed a 10-day investigation into corruption allegations against some of Yushchenko's most senior aides.
Yushchenko ordered the commission to be created after his former chief-of-staff, Oleksandr Zinchenko, resigned two weeks ago and accused former Security Council chief Petro Poroshenko and other top aides of corruption.
The allegations triggered the break-up of the Orange Revolution coalition.
Poroshenko was questioned by prosecutors and gave evidence on Monday, his spokeswoman Irina Friz said on Tuesday.
She said no charges were filed against him, denying reports in the Ukrainian media that he had been charged with impeding the work of the judicial system.
Piskun did not specify whether investigators had opened proceedings against Poroshenko.
But he noted that Yushchenko's two other key aides, Oleksandr Tretyakov and Mykola Martinenko, who had also been accused of corruption, "were not implicated in any wrongdoing".