"Despite repeated high-ranking and persistent demands, Russian authorities have not cooperated sufficiently in the investigation into the murder".
Prosecutors said there were sufficient leads to indicate that the killing was ordered either by the Russian state or by Chechen authorities.
"The foreign ministry has today declared two employees of the Russian embassy in Berlin as personae non grata with immediate effect", the ministry said in a statement.
"We have evidence that a foreign intelligence agency was behind it and therefore the case is going to be taken over by the federal prosecutor this week", an unnamed source told Reuters.
Zelimkhan Khangoshvili, a 40-year-old Georgian national, was shot twice in the head at close range in Kleiner Tiergarten park on 23 August, allegedly by a Russian man who was arrested shortly afterwards.
The victim, Khangoshvili, was a Georgian national, who had fought against Russian Federation in the second Chechen war in 2001-2005, and had fled to Germany after surviving two attempts on his life in Georgia in 2009 and 2015. After the war, he reportedly worked in both Ukraine and Georgia against Russian interests.
He was carrying a Russian passport which identified him as Vadim Sokolov, but German prosecutors on Wednesday confirmed that they now believe that is a false identity. He had previously survived multiple assassination attempts and continued to receive threats after fleeing to Germany. His asylum application, however, was denied and he was slated for deportation.
The case threatens to worsen already heightened tensions between Berlin and Moscow.
The killing has been compared with the poisoning of former Russian agent Sergei Skripal in Britain previous year with a nerve agent, widely blamed on Russian intelligence.
That attack was also blamed on Moscow - leading to the expulsion of dozens of Russian diplomats and triggered a diplomatic crisis between the United Kingdom and Russia.