Mujib Rahman Ansari died alongside his brother, members of his security detail and civilians gathered for prayers in a suspected suicide blast, officials say.
It is not yet clear who is behind the attack, which the Taliban denounced as "sinister" and "cowardly".
The killing is the latest in a string of assassinations of prominent pro-Taliban figures in the country.
According to police, Ansari was arriving at Gazargah mosque to lead Friday noon prayers when a suicide bomber kissed the cleric's hand and detonated an explosive device.
Unverified images on social media appear to show a number of bloodied corpses lying amid a scene of devastation outside the mosque compound in the western Afghan city.
Local officials say at least 23 people were wounded in the attack, but local reports suggest the total casualty figure could be significantly higher.
"Unfortunately, the country's popular religious scholar Mawlawi Mujib Rahman Ansari has been martyred in a cowardly attack during the Friday prayers in Herat," a Taliban spokesman tweeted.
The cleric was well-known in Afghanistan for his support of the Taliban's rule.
Speaking earlier this summer at a religious gathering in Kabul, he called for those who committed "the smallest act against our Islamic government" to be beheaded.
According to AFP news agency, he said "this [Taliban] flag has not been raised easily, and it will not be lowered easily."
No-one has claimed responsibility for Friday's attack, but the militant Islamic State (IS) group has previously issued videos threatening the pro-Taliban cleric.
The security situation in the country, which had improved after the end of fighting following the Taliban takeover, is seen to be deteriorating.