LONDON—The new U.K. government managed to stop recent anti-immigration riots by swiftly arresting and prosecuting hundreds of people who took part in a 7-day spree of violence that included mobs damaging mosques and trying to set fire to hotels housing migrants. Among those arrested and charged in televised trials: a 69-year-old retiree and a 12-year-old boy.

The spreading riots were a baptism of fire for Prime Minister Keir Starmer , who took power last month as his Labour Party won in a landslide. The former public prosecutor is largely credited for a response that quickly reasserted law and order on the streets.

For Starmer, however, now comes a far harder task: addressing the root causes of the riots, in particular, delivering on promises to bring down near-record levels of both legal and illegal immigration, both of which have risen sharply in recent years and turned the issue of immigration into a top voter concern at a time of weak economic growth.

People hold signs at a protest against racism outside Reform UK’s headquarters in Westminster, in London, Britain, August 10, 2024. REUTERS/Belinda Jiao

Following the riots, a YouGov poll found more than half of Britons picked “immigration and asylum” as one of the top issues facing the country—almost equaling worries about the economy. That is the highest level of concern about immigration since 2016, according to the survey, shortly after the U.K. voted to leave the European Union in part due to worries over the rise in immigration from Eastern Europe.

In the years since Brexit, however, immigration has risen further—despite repeated promises by Conservative governments to bring it down. That failure has helped the rise of the anti-immigration Reform UK party, which took 14% of the vote in the recent U.K. election, mirroring a rise in support for anti-immigration and far-right parties in France, the Netherlands and Germany.

Since 2000, the percentage of the U.K. population that was born outside the country has doubled to 17% —higher than the 14% for the U.S. More than 40% of London’s nine million residents were born abroad.

“There are many regular folk who don’t want too many people coming in. That doesn’t make them racist,” said Trudy Lee, a 71-year-old retiree who voted for Reform.

The rioting began after the fatal stabbing of three young girls in Southport, near Liverpool, was falsely attributed online to a Muslim immigrant who had entered the country illegally. The scenes shocked Britain: a mob trying to set fire to a hotel believed to be hosting migrants; the attack of a car carrying two Filipino nurses to work; a Nigerian migrant who came to Britain legally to work as a nurse tearing up on television as he found his car had been torched.

Taken together, it amounted to the biggest outburst of racially fueled anger in Britain since the 1980s and made many of the country’s minority groups wonder how welcome they really were in a normally tolerant and increasingly diverse society. Starmer attributed the violence to “far-right thuggery.” Britain has long had a small minority of far-right groups, famously including soccer hooligans and usually in the most economically deprived parts of the country.

But some political analysts say it would be a mistake to only blame racism or ignore some broader support for the rioters. While most Britons opposed the violence, 7% said they backed the unrest either slightly or strongly, and many more worry about the speed of immigration and the country’s capacity to assimilate migrants, according a recent YouGov survey. That survey also showed 57% of respondents said immigration was “too high,” with 24% saying it was “about right” and only 4% said it was “too low.”

Unlike the U.S., Britain hasn’t historically been a nation of immigration. For several decades until the 1990s, as many people left as arrived—keeping net migration about zero. But starting that decade, those arriving began to outnumber those leaving, a process that accelerated as Britain drew in more legal migrants from around the European Union, from Polish construction workers to Croatian waiters. Around the time of Brexit, net migration was just over 300,000 a year.

Brexit was expected to reduce net legal migration given that most EU citizens would no longer be able to live in the U.K. without a visa. But while Brexit gave the U.K. more control over who it decided to let in, the visa rules it eventually drew up allowed far greater numbers of people to enter from other parts of the world. The upshot: Legal net migration grew quickly, reaching a record level of 764,000 in 2022 before falling back slightly to 685,000 last year, according to government statistics.

That was despite an official government pledge by the Conservatives to keep net migration numbers below 100,000.

In the past few years, some of the new arrivals were part of a humanitarian program to help displaced Ukrainians and citizens of the former British colony of Hong Kong. But that total was just 102,000 last year. Far more numerous were some 616,000 work visas and another 606,000 student visas, according to government figures.

The previous Conservative government tightened visa rules, making it harder for those arriving to bring family members. That means legal migration numbers should start to come down this year and in the following years, experts say. The humanitarian programs for Ukraine and Hong Kong are also winding down.

“The Starmer government is going to inherit a fair wind on the legal migration numbers,” said Rob McNeil, deputy director at the Migration Observatory at the University of Oxford.  But it will have to try to balance the reduced figures with potential damage to the economy from labor shortages in areas like care for the elderly, he said.

The more difficult part will be illegal immigration, which has also risen in recent years. In the past decade, the British and French governments have managed to largely stop migrants from coming through the Channel tunnel hidden in cars and trucks by boosting security on the French side. That has made large numbers of people take to the English Channel to cross in inflatable rafts .

Those numbers have risen fast, from just 299 people in 2018 to a record 45,774 in 2022 and 29,437 last year. This year’s crossings are on pace for a record. The Starmer government has ditched the previous government’s plan to send asylum seekers to Rwanda and said it would focus on breaking up criminal gangs that ferry would-be refugees. It also plans to speed up the processing of asylum seekers once they are here.

The locations of the recent riots correlated closely with both the country’s most deprived working-class areas and zones where migrants were being housed in hotels awaiting their asylum decisions, said Sunder Katwala, head of British Future, a group that monitors social attitudes to migration and integration.

Katwala said the right-wing response to the riots is to claim it is all about immigration, and the left-wing response is to claim it is all about poverty, making people blame someone else for their problems. Both are legitimate conversations to have, while also making clear that violence and racism are unacceptable, he said.

For the most part, Katwala said, integration in the U.K. has gone well. Compared with France, immigrants in the U.K. are far less residentially segregated and have educational achievement closer to the native-born population. Meanwhile, the percentage of Britons who say you have to be white to be British has fallen to just 3% from 10% a decade ago, he said, citing a survey by Ipsos.

Write to David Luhnow at david.luhnow@wsj.com