The earthquake happened about 09:15 local time on Wednesday at Mansfield, not far from the Victorian state capital.
Footage on social media showed damage to some buildings, but no injuries were immediately reported.
The earthquake was also felt in neighbouring South Australia and New South Wales (NSW).
Victoria's State Emergency Service warned residents to be alert for possible aftershocks.
"If you are located in Victoria, you are in danger. Expect aftershocks, stay away from damaged buildings and other hazards. Avoid driving, except for emergencies," the authority said.
On social media, people posted pictures of damaged buildings and rubble in Melbourne.
Several high-rise towers and a city hospital were also evacuated, local media reported. Some city tram lines had been suspended due to damage.
South Australia Premier Stephen Marshall said his state had felt aftershocks. One federal MP, Michael McCormack, said the "ground [was] shaking violently" in Wagga Wagga, NSW.
The Australian Broadcasting Corporation shared a clip of its breakfast TV presenters reacting in the studio as the earthquake hit.
"Is it an earthquake or a structural thing?" presenter Michael Rowland can be heard asking producers. "Let's go… that was a big one."
Large earthquakes are considered an uncommon occurrence in Australia, because the continent lies centrally on a tectonic plate.
This quake - one of Australia's largest in recent years - struck at a depth of 10km (6 miles), according to government tracker Geosciences Australia.
Melbourne, the country's second largest city, is home to about five million people.
The city is currently in lockdown due to a recent wave of coronavirus cases.