Trump also lauded Tillerson, with whom he has long clashed, and wished him and his family well.
The president also told reporters Tuesday that he made the decision to oust Tillerson "by myself” and added that Tillerson will be "much happier now.”
Here are seven things to know about Pompeo:
He was born in California
The 54-year-old native of Orange, California, was born Michael Richard Pompeo on Dec. 30, 1963.
He is married to wife Susan of Wichita, Kansas. They have one son, Nick, and two dogs.
Pompeo graduated from West Point and Harvard Law School
Pompeo attended West Point and served as a cavalry officer in the Army from 1986 to 1991.
After graduating from Harvard Law School, where Pompeo became editor of the Harvard Law Review, he ran two businesses in Kansas.
He’s a former U.S. congressman
Pompeo was elected as representative in Kansas’ 4th District in 2010 and served for three terms before heading the CIA.
According to U.S. News & World Report, Pompeo's 2010 campaign was backed by the political arm of Koch Industries, an energy conglomerate based in Wichita, Kansas (Pompeo's district).
The top officials of Koch Industries, billionaires David and Charles Koch, are influential backers of conservative causes.
Founder of Thayer Aerospace and Private Security (now Nex-Tech Aerospace)
In 1998, Pompeo and other West Point alumni helped found Thayer Aerospace, which he ran with investment funds from Koch Industries. He sold the company in 2006 and became president of Sentry International, an oilfield equipment company.
He's a Tea Party advocate with fiscal and socially conservative stances Considered a Trump loyalist, Pompeo has in the past played down the extent of Russian interference in the 2016 presidential election, but is known to be more hawkish on Russian President Vladimir Putin than Trump.
Pompeo was also highly critical of the Obama administration and its nuclear deal with Iran. He also opposed the Affordable Care Act, shutting down the prison at Guantanamo Bay and accused former secretary of state Hillary Clinton of intentionally misleading Americans about the nature of the attack while serving on the House Select Benghazi Committee.
Following the deadly 2013 Boston marathon bombing, Pompeo was accused of Islamophobia for suggesting some Islamic faith leaders may be encouraging terrorist attacks, according to the BBC.
"When the most devastating terrorist attacks on America in the last 20 years come overwhelmingly from people of a single faith and are performed in the name of that faith a special obligation falls on those that are the leaders of that faith," he said. "Instead of responding, silence has made these Islamic leaders across America potentially complicit in these acts and, more importantly still, in those that may well follow."
He was nominated by Trump for CIA director in November 2016
Trump’s pick for CIA director made headlines in 2016 because Pompeo, who has long been known for his conservative ideologies, would be filling a non-partisan role as CIA director.
He was confirmed to the post by a 66-32 vote and sworn in on Jan. 23, 2017.
Rumors pointed to Pompeo replacing Tillerson
In November 2017, the New York Times reported that Tillerson, who had a rocky relationship with Trump, was on the way out, and Pompeo would be the likely replacement.
Tillerson dismissed the notion as “laughable” as Pompeo remained silent.
The Associated Press contributed to this story.