Boosting manufacturing is critical to India becoming an economic powerhouse. It now has a fresh shot at that with the election of Donald Trump , whose promise to levy sky-high import tariffs on Chinese goods could send more manufacturers to the South Asian nation.

But first India has to get out of the way of its factory owners.

In the southern city of Bengaluru, A. Dhananjaya , who has run a garment manufacturing company for nearly three decades, says he grapples with high labor costs and hundreds of labor-compliance rules. He wouldn’t dare grow beyond about 100 workers, because that would mean more forms to fill out, more licenses to apply for, and more expenses.

Before the pandemic, he lost business to China. In the past two years, larger and cheaper Bangladesh factories have wooed away many customers.

“The collapse is real,” said Dhananjaya, who has laid off half his workers in recent years. “Five years down the line, I’m not sure if the garment industry will survive.”

India has generated splashy headlines under Prime Minister Narendra Modi by wooing high-profile companies such as Apple that have sought to diversify production away from China since the pandemic. But economists say that the country has done little to clear away hurdles for labor-intensive manufacturing, the foundation for economic growth from China to South Korea.

In India, manufacturing’s contribution to its gross domestic product has fallen from around 17% two decades ago to 13% in 2023, according to World Bank data. The country has about 65 million manufacturing jobs, while four times as many people work in agriculture . Millions of young people join the workforce every year in the world’s most populous nation.

That failure is especially visible in its apparel exports, where India, with its history of skilled textile work and vast labor supply, should have an edge. Instead, its annual apparel exports have declined more than 11% compared with a decade ago, clocking in at $15 billion in 2023, according to the World Trade Organization. Over the same period, Bangladesh saw its garment exports grow by more than 50% to $38 billion, while Vietnam’s exports crossed $30 billion.

The two countries top the list of nations that have been able to seize an advantage from China’s declining share in global exports of low-skill manufacturing, according to the World Bank. India doesn’t even figure in the top five.

“India is punching below its weight for labor-intensive manufacturing,” said Trinh Nguyen , emerging market economist with Natixis. “It is an unsexy part of industrialization, but a country that large cannot completely ignore it.”

India’s apparel exports have faced friction from some factors beyond its control—such as global trade policies meant to help the poorest countries. Bangladesh’s clothing exports enjoy duty-free access to the world’s biggest apparel buyers, the U.S. and Europe, an advantage that it has used to become the world’s biggest apparel exporter after China.

But economists and manufacturers say that countries such as Bangladesh and Vietnam have also done a far better job of smoothing the way for firms than India where regulations, especially around labor, discourage companies from expanding.

Factories in India with over 100 employees require government permission to fire workers. Those with at least 50 female workers must set up an on-site nursery. Adding a second shift to turnaround large orders quickly also requires prior government approval.

Dhananjaya said he has kept his full-time workforce small to avoid regulations that require going in person to the local labor department to obtain an updated labor license every time a factory wants to expand. They require proof that the facility has staff trained in first aid and bathroom facilities to handle the extra workers.

His manager devotes a large portion of time to keeping books and updating dozens of different labor registries, including overtime, accidents and salaries. He pays his workers about 45% more than the prevailing minimum wage in neighboring Bangladesh.

“Labor is just a headache,” he said. “It adds costs that make us less competitive.”

Many states bar or restrict women from working in factories after 7 p.m., while male workers increasingly favor gig work or making deliveries over factory shifts. That limits manufacturing facilities to only one shift a day.

Sameer Yadav , the 42-year-old owner of a garment factory in Bengaluru, said his costs would have been far lower if he could add a late shift to an existing factory. Instead, he set up a second factory in 2016 to meet deadlines for large orders.

That doubled his expenses for machinery, supervisors, security guards and permits. Over the past decade, he has lost about 60% of his business to rival countries that can produce shirts for half the cost.

“All the business has gone to Bangladesh,” Yadav said.

Economists and policymakers say it is a pattern that is repeated across many parts of India’s economy that have the best chance of employing lots of workers, from furniture to shoes.

In contrast, Bangladesh has streamlined its permitting process by handing over some regulatory powers to the country’s main trade group, the Bangladesh Garment Manufacturers and Exporters Association.

The trade group issues a same-day permit to factories for hiring additional staff and handles labor department paperwork on behalf of companies, said Shovon Islam, a director of the association. Some permits for importing fabric duty-free are also issued by the group, which acts as a middleman with customs officials.

Nearly 60% of Bangladesh factories have 3,000 workers or more, compared with an average of 150 workers per factory in India.

Islam, who used to operate out of India, said the streamlined regulatory process was partly what pushed him and his Indian business partner to gradually move their factories over the past decade from the southern Indian city of Chennai to Bangladesh.

Now his four factories in the Bangladeshi capital Dhaka have an average workforce of 5,000 people, more than tripling the capacity of his Indian factories, he said. Overhead costs have been slashed 30% by boosting production while eliminating the need for duplicate support staff, storage facilities and transport.

“When you have large factories, the economies of scale are very high,” he said.

Bangladesh’s factories saw several disasters in the rush to grow, but after a factory collapse that killed more than 1,000 people , the industry worked with brands to improve safety.

It takes factories in Bangladesh two or three weeks to produce and ship an order, compared with twice as long in India, according to industry sources.

An overhaul of the labor code by Modi’s government aimed to loosen labor laws, including allowing firms of up to 300 to fire workers without government permission. But they have yet to be widely implemented after facing pushback from labor unions, which organized marches this year calling for the repeal of the laws.

Manufacturing firms in India said they are wary of operating large factories because of the power of organized labor. In September, a strike paralyzed the operations at a Samsung Electronics factory in the southern Indian state of Tamil Nadu for over a month as workers demanded better pay, improved working facilities and recognition of a newly formed union. Apple suppliers including Foxconn and Wistron have also dealt with protests and strikes.

“No manufacturer wants to set up a 10,000 person factory because it really makes them a target,” said Rahul Ahluwalia, founding director of Foundation of Economic Development, a New Delhi think tank. “The politicians will round up your labor and try to extract rents from the businessmen through that.”

India’s failure to sign free-trade agreements with other countries that would slash tariffs on its exports has also made Indian garments increasingly too expensive for global retail companies. At the same time, the country levies high duties on the types of synthetic fabric that factories need to fill fast-fashion orders.

India’s own retailers and manufacturers catering to the domestic market are shifting more production and sourcing to Bangladesh, leading to a sharp jump in India’s apparel imports.

Some firms have managed to grow despite the odds. Garment maker Radnik Exports Global , based in New Delhi, has protected itself by manufacturing high-value garments for buyers such as Hugo Boss and Ralph Lauren , and diversifying into technical fabrics for automobiles and accessories, said Anurag Kapur, a director and member of the founding family.

Kapur expects Trump’s election—and political unrest in Bangladesh which ousted its prime minister in August—to provide opportunities to larger apparel makers like his firm.

But it hasn’t been easy. His family had to scramble in 2019 after Trump canceled India’s special trade status, which allowed for duty-free imports of some garments, and accused India of shutting out American businesses. Trump repeatedly railed over what he saw as India’s protectionist policies, including a 50% import duty on Harley-Davidson motorcycles.

Kapur said some of his garment export orders in production for the U.S. suddenly attracted high single-digit duties. The firm trimmed costs by laying off supervisors and support staff and beseeching suppliers to swallow some of the extra expenses.

“We are like clusters fending for ourselves,” he said. “The government has not looked out for us as an industry.”

If the company had stuck to only manufacturing simple garments, he said, they would be struggling just like 80% of the industry.

“You reinvent yourself, that is the only mode of survival,” Kapur said. “If you haven’t, it’s already too late.”

Write to Shan Li at shan.li@wsj.com