Police spokesman Stefen Petersen said thieves apparently entered through a window about 3:30 a.m. Monday, broke into a cabinet where the "Big Maple Leaf" coin was kept, and escaped with it before police arrived.
A ladder was found by nearby railway tracks.
The three-centimeter (1.18-inch) thick coin, with a diameter of 53 centimeters (20.9 inches), has a face value of 1 million Canadian dollars ($750,000). By weight alone, however, it would be worth almost $4.5 million at market prices.
Petersen would not comment on whether authorities had surveillance video of the crime, but said police assume more than one person was involved because of the weight of the coin.
The museum says the coin is in the Guinness Book of Records for its purity of 999.99/1000 gold. It has a portrait of Queen Elizabeth II on one side and maple leaves on the other.
It was produced in limited quantities by the Royal Canadian Mint to promote a new line of its Gold Maple Leaf bullion coins in 2007. It has been on display at the Bode Museum, on Berlin's Museum Island, since December 2010.
Berlin museums spokesman Markus Farr said the coin is on loan from a private collection, but would not elaborated.
Detectives specialized in crimes involving art are investigating.