P Rajagopal, 71, ordered the murder of an employee because he wanted to marry the man's wife.
He has been battling to avoid jail since he was sentenced in 2009. On Tuesday, he made a final plea on medical grounds but it was rejected.
Saravana Bhavan has 80 outlets across the world and employs thousands.
Rajagopal has been nicknamed the "dosa king" after the South Indian dish which is served at his restaurants. There are branches in major cities such as New York, London and Sydney.
The restaurant tycoon was reportedly determined to marry the wife of one of his employees, based on the advice of an astrologer.
"He was obsessed with her," D Suresh Kumar, a local journalist, told AFP news agency.
Rajagopal was embroiled in scandal in 2003 for attempting to bribe the woman and intimidating her family, which included assaulting her brother.
In 2001, when the woman's husband went missing, she filed a police complaint against Rajagopal. Her husband's body was later found in a forest and police confirmed that he had been strangled.
A local court first convicted Rajagopal in 2004 and sentenced him to 10 years in prison. But this was increased to life in prison by the High Court in Chennai in 2009.
In March, the murder conviction was upheld by the Supreme Court.
When his final appeal on medical grounds was rejected on Tuesday, he turned himself in at the court in Chennai.