The tourists were cruising at 3-151 in the 18th over, before first-gamer Ashton Turner snared the wickets of Asela Gunaratne and Milinda Siriwardana.
Scores were then tied on the last delivery, with Kapugedera pumping his fist in glee after driving Ashton Tye through cover for the match-winning runs.
Sri Lanka scored 5-172 in reply to Australia's 6-168 from their 20 overs.
Top Sri Lankan paceman Lasith Malinga made a successful return to international T20s — he was on a hat-trick late in the Australian innings, finishing with 2-29 from four overs, and also took two catches.
Gunaratne top-scored with 52 from 37 balls, giving him the last laugh in the match.
When Gunaratne came on to bowl, Australian captain Aaron Finch blasted his first ball 20 rows back into the Southern Stand.
Australia made a dream start in the field when Pat Cummins had captain Upul Tharanga caught behind for a duck in the opening over.
It was a bad night for Tharanga — after winning the toss, he dropped a straightforward chance off Travis Head.
After Tharanga's dismissal, fellow opener Niroshan Dickwella and Dilshan Minaweera took control with a blistering stand of 74 from 47 balls.
Legspinner Adam Zampa temporarily blunted the Sri Lankan charge when he dismissed Dickwella (30 from 25 balls) and Munaweera (44 off 29) in successive overs.
Then Turner dismissed Gunaratne and Milinda Siriwardena (15).
Kapugedera made 10 not out and Seekugge Prasanna was unbeaten on eight.
Zampa was the pick of the Australian attack with 2-26 from his four overs.
Finch top-scored with 43, also passing 1000 T20 international runs.