BERLIN—The nationalist AfD scored its first electoral victory in a German state election since its creation 11 years ago, according to projections on Sunday—a political earthquake and a milestone for a continent where centrist parties are increasingly on the defensive.

The AfD was ahead in Thuringia and a close second in Saxony while the three centrist parties forming Chancellor Olaf Scholz ’s embattled government in Berlin were all but wiped out, initial surveys by public broadcaster ZDF showed shortly after the balloting ended.

With a combined population of just over six million, the two eastern states rarely make national news and the results have no practical bearing on the balance of power in Berlin. Despite its score, the AfD is unlikely to end up governing any of the states because it would need to form a coalition with a rival party and most have ruled out working with it.

Yet the vote has high symbolic value in Germany, where no far-right group has won a state or general election since the end of World War II and where centrist parties have successfully kept the AfD out of power since its creation in 2013.

“If you want to honor the wish of voters, governing without the AfD will not be possible any more,” Tino Chrupalla , AfD co-chairman, told German television. “Other parties should think about whether the firewall is something voters are ready to accept.”

Germany has edged closer to other European nations that have struggled to form stable governments as populists of the right and the left have progressed at the ballot box but continue to be ostracized by establishment rivals, leading to hung parliaments and fractious coalitions.

German voters have soured on a conflict-ridden government coalition that has struggled to respond to economic stagnation, inflation, rising immigration, security concerns, and growing fears about an escalation of the war in Ukraine.

Projections by public broadcaster ZDF showed the AfD getting around 33.4% of the votes in Thuringia and 31.4% in Saxony, with Scholz’s center-left SPD scoring under 8% in both states, with its two coalition partners trailing it.

The Sahra Wagenknecht Alliance, or BSW, a populist party named after its leader that the left created in January, obtained 15.5% in Thuringia and 11.5% in Saxony, racing past far more established parties. The party has struck a nerve in the east with its calls to end military aid to Ukraine and with its anti-immigration message.

“There was a big representation vacuum in the German party system,” Wagenknecht told ZDF, saying her voters wanted more diplomatic efforts to end the war in Ukraine, more social justice, less bureaucracy, less immigration, and were against a U.S. decision to station more long-range missiles in Germany. “That’s why we created the BSW.”

The center-right Christian Democratic Union, the biggest opposition party in Berlin, was expected to come in first in Saxony by less than one percentage point and a distant second in Thuringia. It could end up running both states as other parties rally around it and against the AfD to form governments. With no party holding an outright majority, the coalition negotiations could take months.

One of Germany’s deadliest terror acts in recent years, a knife attack by a Syrian asylum seeker a week ago that left three dead and eight wounded and was claimed by Islamic State, was thought to have given the AfD and the BSW, both strongly anti-immigration, a last-minute boost.

“The center-right will decide to what extent an AfD win would be a turning point: So far, they have been relatively consistent in excluding cooperation—more so than in other Western European countries,” said Manès Weisskircher, a political scientist at the Dresden University of Technology.

The AfD win in Thuringia marks a setback for Scholz’s fractious three-way coalition of Social Democrats, pro-market liberals and environmentalist Greens, which has been crippled by internal disagreements. This year, it even came close to failing to agree on a budget.

A survey by the Forsa polling group last week showed less than a third of voters supporting one of the three parties in the coalition. Support for Scholz’s Social Democratic Party, or SPD, stands at 15% nationally.

Pollsters expect the AfD to finish first in Brandenburg, another eastern German state, at an election there on Sept. 22. The party is weaker nationally, however, where it currently polls around 16% following  a series of scandals .

Even outside of power, the AfD is reshaping the political discourse and government policy. After an initial tightening of immigration laws earlier this year, the government announced a raft of new measures last week in reaction to the terrorist attack.

Asylum seekers facing deportation to another European Union country will no longer receive full welfare benefits, Berlin said on Thursday. Knives will be banned from certain public events and train stations, and police will be allowed to use face-recognition software and artificial intelligence to identify terror suspects.

On Friday, the government said it had deported 28 convicted criminals back to Afghanistan on a chartered flight to Kabul, the first time it has done so since the Taliban regained power. Scholz said the government would meet with the conservative opposition in the week ahead to discuss additional measures to reduce immigration, boost deportations and improve security.

Germany recorded 329,035 asylum applications last year, according to the EU’s statistical office—almost half the population of Frankfurt, the country’s financial capital, and the highest annual number since the peak of the European refugee crisis in 2016. Germany is the end destination for almost a third of asylum applicants in the EU.

Such high numbers mean security agencies aren’t able to vet all arrivals. Young men from Syria, Iraq and Afghanistan, in particular, have struggled to find work even when their asylum requests are accepted, and they are overrepresented in crime statistics. People whose asylum applications are rejected are rarely deported.

The suspect in last weekend’s stabbings, identified as 26-year-old Issa al H., arrived in Germany in 2022 via Bulgaria. Under EU rules stating that asylum seekers must submit their claim in their first port of entry in the bloc, he was supposed to be deported back to Bulgaria long before the attack.

However, authorities said they couldn’t find him when they tried to detain him and didn’t make a second attempt. After six months, a legal deadline expired, making Germany responsible for his asylum application.

Write to Bertrand Benoit at bertrand.benoit@wsj.com