As two closely watched murder trials played out in two different states this past week, juries heard strikingly similar stories: men took up guns in the name of protecting the public, and when they wound up killing unarmed people, they claimed self-defense.

In one case, Kyle Rittenhouse fatally shot two men and wounded a third in the unrest following a police shooting in Kenosha, Wis. In the other, Ahmaud Arbery, a Black man, was shot after a pursuit by three white men who said they suspected him of a series of break-ins in the neighborhood. In both cases, the defendants claim they were entitled to start shooting because the victims were trying to take their guns.

“In other words, their own decision to carry a gun became a justification to use it, lest it be wrested away from them,” said Eric Ruben, an expert on the Second Amendment at the S.M.U. Dedman School of Law in Dallas.

For legal experts like Mr. Ruben and others, these two cases expose deep fault lines in the legal and moral concept of self-defense, a doctrine that is particularly cherished in America but ill-equipped to handle an era of expanded gun rights, growing political extremism, violent threats and a strong vigilante strain, all in a country where the perception of threat is heavily influenced by race.

“The problem is that with a citizenry armed with guns, we have blurred every line,” wrote Kimberly Kessler Ferzan, a professor of law and philosophy at the University of Pennsylvania, arguing in the Texas Law Review that a potent mix of “stand your ground” provisions and citizen’s arrest statutes have given people license not just to defend themselves but to go after others. “What is defense? What is reasonable? When may one stand one’s ground and when must one retreat? And, when is a citizen entitled to step in as an aggressor in the name of the state?”

The rifts have surfaced in several debates, beginning with whether openly displayed guns make the bearers feel safer at the expense of everyone else, whether brandishing a gun constitutes a criminal threat or an act of self-protection, and whether people can benefit from self-defense claims if their own actions contributed to the volatility of a situation.

In Michigan, elected officials clashed over whether militia members accused of plotting to kidnap Gov. Gretchen Whitmer were actually trying to make a citizen’s arrest. In St. Louis, there were arguments over whether white residents who pointed guns at Black Lives Matter protesters were committing assault or defending their homes. (They pleaded guilty to misdemeanor assault and harassment, then were pardoned by the governor.)

In a civil trial that began last month against the organizers of the Unite the Right rally in Charlottesville, Va., in 2017, nine plaintiffs who were among those injured or traumatized are arguing that there was a plan and intent to engage in violence, while the defendants say the violence, in which one counterprotester died, was committed in self-defense.

Generally, the law provides a right to self-defense when people reasonably believe there is an imminent threat of harm, whether or not that belief is correct. For example, it may be reasonable to mistake a realistic prop gun for a real gun.

But most statutes say someone cannot claim self-defense if that person was the “initial aggressor” — in other words, if the person provoked the encounter that led to the use of force or was doing something illegal at the time. Initial aggressors may regain their claim to self-defense if they try to withdraw or back away from an encounter. And initial aggressors must actually pose a threat of imminent harm.

“If you assault someone without any justification, that constitutes provocation,” said Cynthia Ward, a law professor at the College of William & Mary. “You do not provoke an encounter if you simply demand an explanation of offensive words, or are talking about a sensitive subject, or engage in an inconsiderate act or travel near someone.”

In the Arbery case, the concept of initial aggressor is complicated by the claim by the defendants — Gregory McMichael, his son Travis McMichael and their neighbor William Bryan — that they were pursuing Mr. Arbery in an attempt to make a citizen’s arrest, and that he was shot only after he tried to grab one of their guns.

Lost in the proceedings is the voice of Mr. Arbery, who conceivably would have had his own claim to self-defense.

“Why is it that the perception is that if Ahmaud Arbery would have complied rather than reached for the gun that he would have lived, and therefore it was incumbent on him to comply?” asked Darrell Miller, a law professor at Duke University and the co-author of a recent paper on the inadequacy of self-defense laws. “Why isn’t the framing that Ahmaud Arbery had three guys, who he didn’t know what their intentions were, rolling up on him with guns?”

Over time in the United States, the tendency has been to expand the right to claim self-defense rather than protect those who may be harmed by misjudgments and mistakes. Castle doctrine laws — also known as “make my day” laws — give people in their own homes the presumption of self-defense if they harm an intruder. The “duty to retreat,” a mainstay of classic self-defense theory, has been eroded by “stand your ground” laws that do not require retreat for people who are in a place where they have a right to be, such as a public protest.

In many states, the burden of proof has shifted from requiring defendants to demonstrate that they acted in self-defense to requiring prosecutors to show that they did not. A new law in Utah gives defendants, except those charged with attacking a police officer, the right to request a special hearing in which prosecutors must prove that the accused was not acting in self-defense.

But while the legal framework for self-defense has expanded, experts say it has left key concepts like “initial aggressor” poorly defined. In the trial of George Zimmerman for the fatal shooting of Trayvon Martin, a Black teenager, in a gated community where he was visiting relatives in 2012, prosecutors argued that Mr. Zimmerman, who is Hispanic, should be considered the initial aggressor. Mr. Zimmerman had identified Mr. Martin as suspicious and followed him both in his car and on foot before Mr. Martin knocked him to the ground. The jury disagreed and acquitted Mr. Zimmerman.

In that case, only Mr. Zimmerman was armed. But who is the “initial aggressor” when two armed strangers face off?

On the evening at issue in the Kenosha trial, Mr. Rittenhouse, now 18, was armed with a semiautomatic rifle that he was not old enough to buy. After he shot and killed a man who he thought was trying to grab his gun, as well as a man who hit him with a skateboard in an apparent attempt to stop him, a nearby paramedic named Gaige Grosskreutz put his hands in the air.

At that point, Mr. Grosskreutz testified this week, he thought Mr. Rittenhouse reracked his rifle and “wasn’t accepting my surrender.”

Mr. Grosskreutz then pointed his own handgun, which he had with him even though his concealed-carry permit had expired, toward Mr. Rittenhouse. Mr. Rittenhouse shot him in the arm.

Mr. Grosskreutz said he was “trying to save my own life.” Mr. Rittenhouse said much the same.

If one sets aside the reasons that brought each man to the parking lot that night, said Renee Jorgensen, a philosopher who has written about self-defense and what constitutes a reasonable mistake, “The way that I would treat that case is that neither wrongs the other, and neither is wronged by the other — a kind of a Wild West situation where it’s not unlike armed combatants in war.”

Of course, both Mr. Rittenhouse and Mr. Grosskreutz — who are white, as were the men who died — had elected to take the law into their own hands to varying degrees, offering themselves that night as private guardians of public safety.

Experts say self-defense, vigilantism and policing are deeply connected — all are deeply racialized American traditions in which Black people, particularly men, are more likely to be viewed as threats and white people are more likely to be given the benefit of the doubt.

In an analysis of homicides done after Trayvon Martin’s death, the Urban Institute found that cases with a white perpetrator and a Black victim were 281 percent more likely to be ruled justified than cases with a white perpetrator and white victim.

In the Arbery case this past week, a defense lawyer objected to having high-profile Black pastors in the courtroom, calling their presence “intimidating.”

Self-defense and police use-of-force cases rely on the same standard of reasonable fear, though officers are given greater leeway than civilians. In police killings, the reasonable fear standard has often been viewed as giving officers cover for implicit bias.

Some scholars are thinking about ways to help self-defense laws adapt to a country awash in guns and counteract pervasive prejudices.

Cynthia Lee, a law professor at George Washington University who is known for her model statute on police use of force, has begun work on a universal definition of what constitutes an initial aggressor. It would add another option to laws like Wisconsin’s, which says the aggressor must have intended to provoke violence with a plan to retaliate, a difficult thing for prosecutors to prove. And it would provide for special scrutiny when guns are involved, whether or not they were legally present.

“If you display a firearm or you point it at another person, that’s a threatening act that ordinarily would give, I think, a reasonable apprehension of death or serious bodily harm,” she said.

Julie Bosman contributed reporting.