Innovation isn’t the first word that comes to mind when you think about a sanitation department.

But a few years ago, when New York City officials found themselves in the market for a better garbage can, they followed a strategy to spark creativity that has worked for centuries, producing breakthroughs from the oceans to the heavens and everywhere in between.

They started a contest.

By setting a goal and engaging the crowd for help with all sorts of tricky problems—driverless cars, missions to space, trash cans—organizations can find novel solutions in places they never would have looked and from people they never would have asked.

Last week, I wrote about the Vesuvius Challenge , a $1 million competition with the goal of using artificial intelligence and machine learning to read 2,000-year-old papyrus scrolls. I’m still thinking about the way a bunch of students pulled off the seemingly impossible—and not just because using modern technology to crack an ancient mystery is undeniably amazing. It’s also because the success of this contest should inspire a lot more contests.

Prize contests encourage talented people to work across disciplines and bring unusual perspectives beyond their areas of expertise. They are especially valuable in areas where progress is hard and unlikely to happen without a push, but not so hard that it’s unattainable. The most effective prize contests are built around a simple premise: This is what we want to do, and we don’t care how it gets done or who does it.

But the real key to prize contests—and the reason there should be more of them—is that they pay only for success. That’s a more efficient model than predicting who is most likely to succeed and giving those people money even if they fail.

They can help with today’s issues and shape the future. And they have already uncovered secrets of the past.

The Vesuvius Challenge happens to be a perfect example of how prize contests should work.

The first thing it got right is that it had a clear, ambitious target: Read four legible passages of 140 characters in the unopened scrolls by the end of 2023. Instead of telling people to do something interesting with AI, people were told to use AI to do this specific interesting thing.

Netflix was similarly precise about benchmarks when the company famously offered $1 million in 2006 to the first algorithm that could beat its recommendation engine by 10%. The company declared a winner in 2009 and paid out the prize, but never used the code.

The Vesuvius Challenge also focused the attention of an entire community on a puzzle that had vexed humanity for centuries. It manufactured an urgent race against the clock by setting a hard deadline. And it gave smart young developers an incentive to compete and collaborate, dangling incremental prizes for those who published their code and shared their critical discoveries with others.

Some of them were driven by money. But many of them just wanted to win. And most of them kept working because they were too obsessed to think about anything else. Silicon Valley has a word for that: They got nerd-sniped.

Napoleon to NASA

Prize contests like the Vesuvius Challenge have existed for so long and resulted in so many unexpectedly brilliant ideas that it’s surprising there aren’t more of them by now.

In the 1700s, when Britain was losing ships to navigation errors, the world’s most powerful country offered piles of money to anyone who could devise a simple way of measuring longitude while at sea. That prize led to the invention of the chronometer.

By the early 1800s, “Napoleon was a great believer in the use of cash prizes to stimulate invention and innovation,” historian Andrew Roberts told me. In the mid-19th century, with France suffering through the national crisis of a butter shortage, the emperor put a bounty on a substitute for butter. Et voilà! Margarine was born. Before long, the French loved competitions as much as complaining.

The early 20th century brought one of the most famous contests of all time, the Orteig Prize, a $25,000 reward for the first nonstop flight between New York and Paris, which went to Spirit of St. Louis pilot Charles Lindbergh. Explorers have been following the money into the skies ever since.

In 1996, the Ansari X Prize floated $10 million to the first private team to launch a crewed spacecraft twice in two weeks, which it paid in 2004. The following year, the National Aeronautics and Space Administration launched a series of prizes to engage the public in developing advanced technology. NASA’s challenges have covered everything from lunar landers to lunar loos , plus astronaut gloves that were stitched together with a sewing machine on a dining-room table.

One of the current NASA competitions is the Deep Space Food Challenge , a $3 million contest to feed astronauts on long-duration missions. Among those experimenting with microgreens and single-cell proteins are university teams, startups and one scientist working out of his garage in Colorado.

“We recognize that we have some of the best and brightest at NASA,” said Angela Herblet, a NASA program analyst and challenge manager. “But we don’t have all of the best and brightest.”

One man’s trash

To understand how federal agencies began giving out hundreds of millions of dollars through contests, you have to flash back to the 1990s, when a policy wonk named Tom Kalil read about Britain using incentive prizes centuries ago and thought: Why isn’t the U.S. government doing this?

It struck him as odd that the country rarely made financial commitments that were contingent on success. So he asked the National Academy of Engineering to study prizes and how recognition prizes (for stuff that somebody has already done) compared with inducement prizes (for stuff that nobody has done—yet). He helped the Defense Advanced Research Projects Agency get the authority for prize competitions like the series of challenges it held for self-driving cars. Then he worked with the Senate in 2010 to make sure every agency could offer prizes up to $50 million each.

Now agencies are hosting competitions and funding their own moonshots through the crowdsourcing platform Challenge.gov . Those prizes allow them to pull innovation from the public rather than push it through contracts and grants.

Of course, prize contests don’t always work, and they’re probably not right for tasks that require expensive hardware, trade secrets or specialized expertise. Still, even the ones that fail can be successful in their own way. The Google Lunar X Prize , a $30 million race to put a rover on the moon, ended in 2018 without a winner, but that pot of money helped push the frontiers of private space exploration.

Meanwhile, back on Earth, there was another contest that began only a few months later.

That’s when New York City’s Department of Sanitation officials decided to give the iconic green wire litter basket a makeover. What they did next was unconventional: They opened the bidding process to anybody—and everybody. The only requirement was an idea for a garbage bin that would make life better for New Yorkers and sanitation workers. And worse for hungry rats.

“I didn’t think we would ever get a chance to work on something like this,” said the leader of the winning team , Colin P. Kelly, a New York industrial designer with a passion for municipal waste and recycling. “We weren’t a firm that would be asked to submit in a traditional procurement.”

That’s because they weren’t a firm of any kind.

“We were literally just a group of friends,” he said.

But something funny happened when they began thinking about garbage: They got nerd-sniped.

They spent time with sanitation workers to learn how they used the existing receptacles so they could build a superior one and make those jobs easier and the streets cleaner. Their modular, three-part design makes the more durable bins lighter and less of a burden to lift. They’re also better at preventing overflow and deterring rodents.

Now these designers, engineers and architects who happened to live in New York and never would have had the opportunity to reimagine trash cans see their work every time they step outside, where the waste and recycling baskets on thousands of street corners are being replaced by theirs.

“I think being able to walk around the city and see them is pretty cool,” Kelly said. “That’s the prize.”

Write to Ben Cohen at ben.cohen@wsj.com