Tillerson, the former head of Exxon-Mobil, had found himself at odds with the President on a number of occasions over the past year. There were stories that he had labeled Mr. Trump a 'moron,' and on more than one occasion, the President seemed to be conducting foreign policy without Tillerson's direct involvement.
A top State Department official told reporters that today's move had come as a surprise.
"The Secretary did not speak to the President and is unaware of the reason, but he is grateful for the opportunity to serve, and still believes strongly that public service is a noble calling," spokesman Steve Goldstein said.
Evidently that tweet - and several others - about Tillerson's departure did not sit well at the White House, as Goldstein was fired a few hours later.
In December, when there were reports of Tillerson's imminent departure, the President said that wasn't happening - but he noted that he was the decision-maker when it comes to U.S. diplomatic actions.
"I call the final shots," the President said then in a tweet.
As news of the changes emerged this morning, the White House sent out a series of statements - none of them were from Tillerson, as reports indicated he had not spoken with the President about the change.
Tillerson's final public statements as Secretary of State - made to reporters on a plane flight back to the United States from Africa late on Monday - directly confronted Russia on a series of subjects, including the nerve agent attack in Britain, Ukraine and Syria.
The President's choice for CIA Director is Gina Haspel, currently the Deputy Director of the CIA. As the President noted, she would be the first woman to serve in that post.
On Capitol Hill, former colleagues of Pompeo in the U.S. House said he would do well in his new post.
"Mike Pompeo will do a great job as Secretary of State," said Rep. Ron DeSantis (R-FL), who said Pompeo was 'smart, tough, and works his tail off.'
"This is such a good appointment," said Rep. Lee Zeldin (R-NY).
While Republicans praised Pompeo, the CIA Director has raised some eyebrows among Democrats, as they looked at the changes in a much different way.
"The chaos in Washington continues unabated. What will be next?" said Rep. John Garamendi (D-CA).