Opposition parties were able to secure 174 votes in the 342-member house in support of the no-confidence motion, the house speaker said, making it a majority vote.
The vote was held past midnight after opposition parties brought a motion against him, which was upheld by the Supreme Court.
Khan had said that there was a US-led conspiracy to oust him.
Pakistan's parliament is set to convene tomorrow (Monday) to elect a new leader.
That person will remain in power until October 2023, when the next election will take place.
With this incident, Imran Khan becomes the first Pakistani Prime Minister to be ousted by a No Confidence Motion.
The Supreme Court ruled on Thursday that Khan, 69, had acted unconstitutionally when he blocked the no-confidence vote and dissolved parliament.
This angered some of the opposition MPs who accused Khan of being a traitor.
Minutes prior to the vote, Khan’s friend the speaker of Pakistan’s lower house of parliament, announced his resignation. Members of Khan’s party (PTI) left the building, insisting he was the victim of an international conspiracy.
In a tweet, opposition leader Shehbaz Sharif said Pakistan and its parliament were “finally freed from a serious crisis”.
Sharif had further stated, “Congratulations to the Pakistani nation on a new dawn.”