The Belarus foreign ministry said it had ordered Mikhail Khvostov, its own ambassador to Washington, to return to Minsk, the capital, in protest against US sanctions on Belneftekhim, Belarus' oil monopoly.
Diplomat to stay
Casey said Stewart will remain in Minsk while the US reviews the situation.
He also said that it is in US interests "to have high-level diplomatic representation in Minsk to express US concerns, including that the government release all political prisoners".
The Belarus government had reportedly invited diplomats to a meeting on Friday and "suggested strongly" that Stewart leave.
She was not allowed into that meeting but was called to a separate meeting later and was told the government "believed that she should leave the country".
The decision was one of a number of tough measures that will be taken in response to the US, which has "violated the agreed algorithm on normalising relations", a Belarus government statement said.
Washington describes the government of Vladimir Lukashenko, the Belarus president, as "Europe's last dictatorship" and blacklisted Belneftekhim, freezing the accounts of its representative office.
On Thursday, local media reported that the US treasury department introduced "limitations on the access of the company's subsidiaries to the US market".
Belarus had in December warned of a retaliation if the US.
Overture to EU
While dealing harshly with the US, Belarus made a friendly gesture to the EU, which has also introduced economic and travel sanctions against Belarus.
The EU has been allowed to open an office in Belarus, a presence long sought.
Lukashenko began indicating a desire for better relations with the West after Russia's decision to sharply raise prices for oil exports to Belarus.
The country's Soviet-style, centrally controlled economy has long been dependent on Russian energy supplies.