Asked earlier on Monday whether his actions were plunging the institution into turmoil, Mr Gaetz told reporters: “You talk about chaos as if it’s me forcing a few votes and filing a few motions.
“Real chaos is when the American people have to go through the austerity that is coming if we continue to have $2 trillion annual deficits.”
Mr Gaetz told reporters outside the Capitol on Monday night that he would be up for supporting Louisiana Republican Steve Scalise – currently deputy to Mr McCarthy – to succeed the Speaker.
Tom McClintock, a California Republican, blasted the “self-destructive course” of trying to remove the Speaker, before Mr Gaetz’s floor speech.
Without naming his Florida colleague, Mr McClintock said: “I implore my Republican colleagues to look past their prejudices, their passions, their errors of opinion, their local interests and their selfish views.”
According to the rules of the chamber, the Speaker is required to keep a list of individuals who could act as a temporary replacement in case the role is ever vacated.
If Mr McCarthy were voted out, this list would be made public and the person at the top of it would be named Speaker pro tempore until elections were held in the chamber for a new leader of the majority party in the chamber.
It would take a simple majority of the House to remove the Speaker in a floor vote – 218 votes, when no seats are vacant.
Republicans control the chamber by a narrow 221-212 majority. But only a handful of hardline Republicans have indicated they are willing to remove Mr McCarthy.
Democrats must now decide if they will step in and vote to help the Speaker keep his job.
Democrats are unhappy with Mr McCarthy after he recently approved the launch of a congressional inquiry to see if there is enough evidence to impeach President Joe Biden.
But left-wing New York lawmaker Alexandria Ocasio-Cortez told CNN on Sunday her fellow Democrats might be interested in politically bailing out Mr McCarthy if they can extract concessions from him.
The rare procedural tool to remove a Speaker has only been used twice in the past century and never successfully.
It was last used in 2015 against Speaker John Boehner.
The motion to remove him failed but it built enough pressure on Mr Boehner that, unable to unite his caucus, he announced his resignation two months later.
Before then it was last used in 1910.