The government had been eager to resolve the bank dispute before a meeting of G7 financial heads in the US later this week.
The issue of who should head the central bank in the world's second-largest economy reached a deadlock after the five-year term of Toshihiko Fukui expired on March 19.
Nominations for bank posts need approval from both houses of parliament.
Expressing disappointment over the outcome, Nobutaka Machimura, a senior government spokesman, said he was unhappy with the battle over the number two post.
"We can't come up with an alternative so quickly, so it's inevitable the deputy's post will be vacant for a while," he said on national television.
Machimura, who is the chief cabinet secretary, said he did not understand why Watanabe was rejected since he had "good knowledge in international finance, administrative skills and a respected personality".
The central bank vacancies are the latest in a series of problems facing Fukuda whose approval ratings have continued to slide over a number of scandals, including millions of missing pension records and a criminal investigation of several defence officials.
The opposition had previously vetoed two other candidates for the banking job, leaving the Bank of Japan leaderless in the middle of a global credit crisis.
On Tuesday Shirakawa, who was approved by both houses of parliament, said he would maintain the central bank's independence and transparency.
"The vacancy of the governor post is clearly an abnormal situation that must be resolved urgently," he said in a lower house hearing on Tuesday.
"I'm determined to devote my heart and soul to fulfil the duty as governor if I can only get your approval."