When asked if he was distancing himself from President Trump after leaving, Spicer replied, "Absolutely not".
Spicer's tense, gaffe-filled time working for Trump came to an end when he resigned in July.
Spicer said President Trump made him address the inauguration size during his first press briefing.
Despite the added turmoil often brought on by Trump's tweeting, Spicer said the president nearly never asked him to look over his post.
"Why is [Trump] so concerned with size - have you ever seen the president naked?" "And saying that the inauguration crowd was the biggest crowd, I think, ever, biggest audience ..." "But the validity of the election compared to looking a photos of the crowd at an inauguration?" "Whether or not you agree or not isn't your job". "But ultimately he's the president, he decides and that's what you sign up to do".
Other notable moments during his tenure found Spicer comparing a Syrian chemical weapons attack to the actions of Adolf Hitler and admonishing a reporter for shaking her head in disagreement while he spoke.
"You can look at a set of facts and come out with one opinion, but someone else can say the facts are the same here and I come out with a different conclusion", Spicer said.
"Right! Exactly!" Kimmel shouted at Spicer. "[Twitter] was one of the president's most effective tools on the campaign trail and he continues to utilize it, and you were constantly kept on your toes".
The long-embattled Trump spokesman also shared how strangers would approach him to thank him for his service, with some telling him: "I didn't vote for the president, I'm not a republican".
Kimmel swung back around to the incident one more time, as they discussed Spicer's relationship with the press, showing a video of the time ABC's Jonathan Karl asked if Spicer meant to always tell the truth.
"I don't think it was probably the best start, no", he told Kimmel.
Kimmel, for his part, did not give him an easy ride, basically calling him "childish" at one point.
"I think it's a two-way street".
The former press secretary said that the president didn't watch all of his press conferences, but would at times provide feedback. "And my view was is that if I'm going to have to partner with somebody that I don't believe had the skill set to execute the job, then it was incumbent upon me to either step aside or make my voice known".
He continued to criticize the press for "protecting" itself, but in general his views toward the people that took him to task on a daily basis are far kinder than those of his former boss, who has said on multiple occasions that the press is the enemy of the American people.
"Lumping everyone together as fake news just seems bad to me", Kimmel replied, equating the idea to people lumping "all priests together as child abusers".
In a wide ranging interview, Spicer and Kimmel touched on Spicer's former role as White House press secretary - a post he left in July - as well as Spicer's feelings toward facts.