The Japanese public will be asked to donate old phones and small appliances to gather two tonnes of gold, silver and bronze for the 5,000 medals.
The project hopes to promote sustainability and reduce costs.
"A project that allows the people of Japan to take part in creating the medals is really good," said Tokyo 2020 sports director Koji Murofushi.
"There's a limit on the resources of our earth, so recycling these things will make us think about the environment."
Collection boxes will be placed in local offices and telecoms stores from April and will remain there until the metal required has been collected.
Members of Japan's Olympic organising committee tabled the idea to government officials and companies in 2016.