The vote left Congress in limbo as it struggles to update farm-subsidy and nutrition programs, pass a government funding bill and consider further aid to Ukraine.
It was unclear who would succeed McCarthy.
McCarthy has repeatedly angered Democrats in recent weeks, including launching an impeachment inquiry into Biden and giving them little time to read a stopgap spending bill to prevent a government shutdown on Saturday, which must be passed. They needed their votes.
The Democrats could have saved McCarthy, but after considering it he said it would not help the Republicans solve their own problems.
Other Republican leaders such as Steve Scalise and Tom Emmer could potentially be candidates, although none have publicly expressed interest.
Representative Patrick McHenry was named to the position on a temporary basis.
The last two Republican speakers, Paul Ryan and John Boehner, retired from Congress after clashes with those on their right wing.
In debate on the House floor, Gaetz and a handful of colleagues criticized McCarthy for relying on Democratic votes to pass the temporary funding, which led to a partial government shutdown.
Republican Representative Bob Good said, “We need a speaker who can fight for something — anything — other than being speaker.”