The inquiry, announced on Thursday, comes in the wake of allegations that the US Central Intelligence Agency was operating secret jails in Romania and Poland and covertly flying prisoners through European Union airports.
Leaders of the political groups in the assembly decided late on Wednesday to set up a temporary committee of inquiry, the mandate and composition of which will be determined next month.
Parliament will have to endorse the committee's remit.
Members of the European Parliament approved a resolution expressing their concerns about the "presumed use of European countries by the CIA for the transportation and illegal detention of prisoners".
The resolution, which also called for the parliament to investigate the allegations, was passed by 359 to 127, with 32 abstentions.
Earlier this week, the EU’s senior justice official urged members of the European Parliament to wait until a probe by the Council of Europe, a human rights watchdog, was concluded early next year before pushing ahead with their own probe.
Condoleezza Rice, the US secretary of state, said on a European tour last week that the United States respects the sovereignty of European countries in its fight against terrorism, but she would not confirm or deny specific reports of CIA secret prisons in Europe.
Franco Frattini, the EU justice and security commissioner, told parliament on Wednesday that there was no evidence so far to confirm the allegations.