The former Wall Street financier had drawn criticism after calling a reporter to give a profanity-laced tirade against his colleagues.
Mr Trump's chief of staff Reince Priebus and spokesman Sean Spicer both left their posts with his appointment.
The decision was made by Mr Trump's new chief of staff, Gen John Kelly, who was sworn in on Monday.
The president was also unhappy with Mr Scaramucci's performance, the White House confirmed.
Mr Trump's spokeswoman, Sarah Sanders, said the president thought Mr Scaramucci's comments to the reporter "were inappropriate for a person in that position".