He was being treated for age-related illnesses in Delhi's AIIMS hospital.
Vajpayee was one of the founding members of the Hindu nationalist Bharatiya Janata Party (BJP), which currently governs India.
Considered a giant of Indian politics, he served as prime minister three times between 1996 and 2004 and was instrumental in making India a global nuclear power.
During his second stint as prime minister in 1998, he surprised the world by announcing that India had conducted a series of underground nuclear tests.
It was the first time India had carried out nuclear tests since 1974.
The experiments had taken place without any warning to the international community, which led to widespread outrage and concern.
But in India he was hailed as a hero, and it helped him bolster his image as a leader who was serious about national security and not afraid of international pressure.
Prime Minister Narendra Modi led condolences in a series of tweets, calling Mr Vajpayee's death a "personal and irreplaceable loss."