Canadian Foreign Minister Bill Graham made the announcement on Wednesday after his Iranian counterpart, Hamid Reza Asefi, said the trial was an internal issue.
"The request for the presence of Canadian observers is contrary to all international principles and regulations and is unacceptable," Asefi said on state television.
Graham, slamming what he said was "completely unacceptable behavior" by Iran, said Ottawa had been promised that three Canadian observers could attend the trial.
Zahra Kazemi, 54, a Canadian photographer of Iranian descent, died a year ago after she was arrested for taking pictures outside Tehran's Evin prison.
"It's a complete rejection of the rule of law ... justice will not be done behind closed doors in Iran," Graham told reporters in Ottawa.
"I am calling in the [Iranian] ambassador this afternoon to make that point to him and I will be recalling from Tehran our ambassador ... to examine what future actions we can take," he said after talks with Kazemi's son Stephan Hachemi.
In the immediate aftermath of Kazemi's death last year, Ottawa pulled out ambassador Philip MacKinnon, but he returned.
Graham downplayed the idea of slapping sanctions on Iran, saying this would only hurt ordinary Iranians.
Kazemi was laid to rest in Shiraz,
a southwestern Iranian city
"It's time to consider everything ... we have to look at what can be effective," he said when asked what Ottawa could do.
An Iranian Intelligence Ministry employee, who prosecutors said caused the blow to the head that killed Kazemi, denied the charges on the first day of his public trial last October.
The trial adjourned after one day. Iranian Nobel Peace Prize winner Shirin Ebadi has said she will represent Kazemi's family.
Iran's judiciary initially said Kazemi died of a stroke. But a government inquiry said she died of a brain haemorrhage caused by a heavy blow while she was in custody.
In a reflection of deep-seated rivalries between Iran's hardline judiciary and reformist government, the Intelligence Ministry has repeatedly called the trial a sham and accused the judiciary of ignoring evidence pointing to other suspects.