The prosecutors wrote that going after lawyers who wrote non-binding recommendations for the president and his senior staff, rather than targeting higher-ranking officials who authorised the alleged torture, "raises important problems from a legal standpoint".
They also questioned the appropriateness of a case that would effectively put on trial "all of the policies of the past US administration," saying such an endeavour would go beyond the scope of the Spanish legal system.
Those blamed for giving legal cover for torture include Gonzales, Douglas Feith, ex-undersecretary of defence, and David Addington, the chief of staff of Dick Cheney, the former vice-president.
Others are John Yoo and Jay S Bybee, both justice department officials, and William Haynes, a Pentagon lawyer.
The complaint alleged that the men, nicknamed "The Bush Six," cleared the path for torture by claiming in advice and legal opinions that the president could ignore the Geneva Conventions, and by adopting an overly narrow definition of which interrogation techniques constituted torture.
Bush has steadfastly denied the US broke the law.
|Baltasar Garzon asked prosecutors to advise on whether to proceed or not [EPA]
But Washington has acknowledged that Khalid Sheikh Mohammed, the self-described plotter of the attacks on the US on September 11, 2001, and other prisoners were waterboarded.
The case is one of several legal actions taken or being considered against Bush administration officials overseas, and the first to get this far.
It comes after assurances to CIA operatives by Barack Obama, the US president, that they would not be prosecuted for using waterboarding and other harsh forms of interrogation on suspects.
Eric Holder, the US attorney-general, also offered them legal help if anyone else takes them to court over methods approved during Bush's presidency.
The Spanish case was brought to Baltasar Garzon, Spain's leading investigative judge, last month by a group of human rights lawyers.
According to Spanish custom, Garzon asked prosecutors to recommend whether to proceed.
In Friday's writ, the prosecutors recommended that if any investigation is in fact opened, Garzon should be replaced by another judge who is already investigating whether secret CIA flights to or from Guantanamo entered Spanish airspace or landed at Spanish airports.
The move will put tremendous pressure on Garzon to step aside, or at the very least allow a board of judges at the National Court to decide the matter.
Observers say the removal of Garzon would be another serious blow for the human rights lawyers, who saw him as being sympathetic to their cause.
Prosecutors said on Friday that they would ask the attorney-general for more clarity on whether and when to move forward in other universal jurisdiction cases in the future.
In his comments on Thursday, Conde-Pumpido said a case against the US officials would have turned Spain's national court into "a plaything" for competing political interests.