On Tuesday, the FSB released a video it said showed Grecu picking up military secrets at a bag drop.
The Romanian foreign ministry accused Russian authorities of a "serious violation" of diplomatic protocol by detaining their diplomat.
"The Ministry of Foreign Affairs condemns the serious violation by the Russian authorities of the Vienna conventions on diplomatic protocol of 1961 by the detention during the day of August 16 of a Romanian diplomat accredited to Moscow and the completely inadequate treatment to which they subjected him," it said in a statement.
The ministry said Grecu would leave Russia within the 48-hour deadline set by Russian authorities.
Relations between Romania and Russia have become increasingly tense since Bucharest's decision to host three batteries of interceptor missiles as part of a planned US defence shield in Europe.
In February, Russia said it has "serious questions regarding the true purpose of the US missile system," and denounced the lack of transparency by Bucharest and Washington on the issue.
Romanian and US officials have attempted to reassure Moscow several times that the system is purely defensive.