Through history, a party's control of the House of Representatives has not changed in the middle of a session of Congress, though majorities have been slim before.

But with a razor-thin GOP majority in the House and a threat to oust Speaker Mike Johnson, could control of the House flip to Democrats ahead of the general election?

Since the start of 118th Congress, Republicans have maintained control of the House by only a handful of seats. In recent months, there has been a slew of retirement announcements and even some prominent Republicans announced they are leaving without notifying leadership before the end of their term -- further reducing GOP control.

"Every conservative is frustrated about our current position," Johnson said on Fox News' "Hannity" earlier this week. "We have the smallest majority in U.S. history. We have a one-vote margin right now so it's very difficult to move big things up the field for our principles."

Control of the House could come down to the numbers and if more Republicans decide to leave before their term ends.

Former Republican Rep. Ken Buck of Colorado resigned on March 22 and House China Select Committee Chairman Mike Gallagher will leave Congress early on April 19. His seat will remain empty until after the 2024 election.

In a self-inflicted wound earlier this year, the House voted to expel former Republican Rep. George Santos from Congress, decreasing the majority. Now the embattled congressman, who plans to run for New York's 1st Congressional District, said last week that he will run as an independent.

The current makeup in the House is 218 Republicans to 213 Democrats, a difference of five seats. Once Gallagher departs, the breakdown is 217 Republicans to 213 Democrats, meaning Republicans can afford only one defection on votes to pass GOP-backed measures.

The special election to fill New York Democratic Rep. Brian Higgins' seat is April 30, which is likely to give Democrats another seat. At this point, there would be 218 Republicans and 214 Democrats. But on May 21, there will be an election to fill former Speaker Kevin McCarthy's reliably red seat.

"It's really quite unprecedented the number of retirements that have been announced in both parties, but particularly among the more seasoned Republicans in the House," said Linda Fowler, professor emeritus of government at Dartmouth College.

"I'm sure there's enormous pressure on remaining Republicans to stand their ground, not do any more retirements and so forth," Fowler added.

Karen Hult, a political science professor at Virginia Tech, said the likelihood of a flip from Republican to Democratic control in the House decreases as the general election nears.

"I don't think it's impossible. I do think it's unlikely because I think as we move closer to the election, and people are listening to their folks back home, a lot of people on the Republican side are saying, 'You know, we're going to be listening to nominee Donald Trump on the Republican side, we're also going to be listening to members of our district. But we need to be careful to that we don't give the house to the Democrats,' which could happen," Hult said.

Fowler, who specializes in American politics and Congress, said narrow House majorities have not always been the norm.

"For most of the history of the Congress, one party has been the majority, it's been a dominant majority, and its majority has lasted a long time in the House of Representatives," she said -- adding that both Republicans and Democrats have enjoyed seasons of strong majorities.

That changed with the help of Republican Newt Gingrich, who helped bring his party the majority in the House after 40 years in the minority. In 1994, Republicans took control of the House during the midterm elections thanks, in part, to Gingrich's push for a "Contract with America," a conservative legislative agenda.

That year, the GOP took control of the House and Senate -- and Gingrich was named the speaker. Under his speakership, what Fowler called the "Gingrich revolution" happened – "where committee chairs were pretty much brought to heel … and undermined, staff resources were shifted to the party leaders" and the speaker decided "who sits on which committees, who the chairs are going to be, and what comes to the floor or not."

It has been "highly unusual" to have such slim majorities over the last few elections, Fowler said.

With small majorities comes more polarization, Hult said.

"What I think is more interesting or unusual at this point, is that it really hasn't been since really then the 1890s, that we've had this much party polarization in the country," she said.

"When the majorities get tighter, close, smaller, polarization goes up, because the stakes are so high that everything, even procedural votes, becomes looked at by the members in terms of the next election," Fowler said. "And so with these really tight majorities, it's really hard to get anything done."

'Difficult' path with passing legislation

The slim GOP majority has made it increasingly difficult to pass bills with only Republican votes, leaving Johnson leaning on Democrats to get some legislation across the line.

Johnson said he hopes the conference will unify to pass legislation, but acknowledged the challenge given the current dynamics.

"We've got to unify -- we've got to unify when you have such a small majority," Johnson said on "Hannity." "If the Republicans will stand together, we have a much better, obviously, negotiation strength when we're dealing with Democrats in the Senate and a Democrat in the White House."

Unification has not always been the name of the game as Johnson works to appeal to Republican hard-liners in his conference.

Hult predicts there will be few issues where Republicans and Democrats will band together to move legislation out of the House.

As former President Donald Trump puts his finger on the scale, weighing in on legislation such as the $95 billion foreign aid package -- it will be more and more challenging to compromise, Hult said.

"I think it'd be very difficult to find that narrow path in which they can find something to agree on," she said.

Speaker Hakeem Jeffries?

There is an effort to remove Johnson led by Rep. Marjorie Taylor Greene. The Georgia Republican filed a motion to vacate the chair before recess, placing intense pressure on Johnson.

Greene filed the motion to vacate Johnson after a vote to fund the government to prevent a shutdown -- which Johnson needed Democratic votes to pass. Greene called her motion to vacate a "warning," adding that "it's time for our conference to choose a new speaker."

"If Republicans once again send the House into disarray by moving to vacate the Speaker, the question is not what Democrats can do to save Speaker Mike Johnson from the extremists controlling his party, but instead whether there are two Republicans who will put country over party to provide the necessary votes to make Hakeem Jeffries the Speaker of the House," New York Democratic Rep. Dan Goldman said in a statement.

Asked about the possibility of a historic change to Speaker Jeffries, Johnson shot it down Monday.

"Look, that's a risk, but I don't think that's going to happen," he said, later adding "Hakeem is not going to be speaker and we'll take care of business."

Hult said she doesn't think it's likely that we will see a Speaker Jeffries before the general election, but left a window open on the possibility saying it "certainly it could happen."

"Jeffries has been really skillful in keeping the party together, and if the allure of maybe having control of the House starts moving some of those members to say, 'Well, let's give it a try and see what happens,' they may well be pulled back by Democrats and think, 'Wait a second, I'm concerned about my own district and my own reelection, but I'm also concerned about being able to reelect President Biden,'" Hult said.

Many voters want to see a sense of decorum restored in a Congress riddled with chaos, Hult said.

"What's clear to a lot of us from the outside, in looking, especially at the House, and a little bit increasingly at the Senate, is that there has been a clear erosion of norms of civility, and of trying to at least work together despite some serious disagreements, that's going to be really hard to recover," Hult said.

"And it's not going to happen in between now and November."