In a statement, Schwarzenegger said that the show's 'baggage' was the reason behind his decision to quit.
The show, which Donald Trump used to host, concluded its latest series with the poorest ratings of any of its celebrity incarnations.
Schwarzenegger accused the controversial US President of being the reason viewers are shying away from the show, saying that Trump's continued involvement has left 'bad taste.'
Despite being President, Trump has remained the show's executive producer.
"People don't want to participate as a spectator or as a sponsor or in any other way support the show," Swarchznegger said in an interview.
The actor added that the United States is in a very divisive period, and he believes that the show got caught up in all the division that the Trump Presidency has brought.
In January, President Trump taunted Schwarzenegger over the show's poor ratings. In a series of tweets, Trump referred to himself as a 'ratings machine,' adding that Schwarzenegger got 'swamped' by comparison.
Trump later voiced his belief that the show has been 'a total disaster' since Schwarzenegger took over. The actor responded by suggesting they swap jobs.
In his statement, the actor added that he had enjoyed working with the NBC network and the show's executive producer Mark Burnett.
The former California Governor applauded the show's crew, celebrities, and marketing department, saying it was 'a straight ten.'
He added that he would love to work with them again, on a show that 'doesn't have this baggage.'