Michelle Shahbazyan was excited to give her beloved older relatives the scrapbook she’d made for Christmas last year, with family photos and pictures drawn by her kids.

That was until she watched the couple unwrap her cousin Stella’s gift at the family’s holiday gathering: a three-week European cruise, first-class airline tickets and matching luggage.

“I was embarrassed,” says Shahbazyan, 42 years old, a life coach in Scottsdale, Ariz. “No matter what you do, she will always trump it.”

Brace yourself for the real family drama this holiday season: navigating the gift-giving quagmire.

Finding time to shop (and figuring out how to pay for it all) are the least of our worries. We also have to face the awkwardness of giving a bad gift. The minefield of pleasing picky in-laws. Irritation when a sibling shows up with better gifts. And—horrors!—shame when the look on our teenage niece’s face tells us we’re not as cool as we thought.

So much for the most wonderful time of the year.

Exchanging holiday gifts with family members is fraught. And all the stress and competition is made worse when presents are exchanged at family gatherings—everyone can see who gave what.

“Gift giving is supposed to be about making people happy,” says Evan Polman, an associate professor of marketing at the University of Wisconsin-Madison’s business school. “To be upset with it suggests you’re being like Scrooge.”

Not everyone agrees on what makes a good gift. Givers prefer to give unexpected gifts that make a big impact in the moment, often ignoring the wishes of recipients, who tend to want something they asked for or something useful, research shows. Givers also spend more money on gifts than recipients would like, which can make them feel indebted.

Often, givers use presents to try to influence the recipient, such as when your super-fit brother-in-law who is always urging you to join CrossFit gives you a barbell for Christmas.

“Gift giving in families can be an act of micropolitics,” says Chihling Liu, a senior lecturer at Lancaster University in the U.K., who studies the dark side of gift giving. “They’re trying to assert their identity on each other.”

In addition to giving a cruise to her relatives last year, Stella Derrostomian gave her cousin Michelle, the scrapbooker, a pavé diamond bracelet. She gave Michelle’s children huge bags stuffed with gifts, including fancy dresses and books for her daughter and Formula One clothing and a remote-control car for her son.

Michelle says she loves her cousin and appreciates the gifts. But she can’t keep up. “Every time she hands me a gift I ask: ‘Why are you spending so much?’” she says.

“I love to see the joy on my family members’ faces,” says Stella, 64, a real-estate agent in Los Angeles. “And I can’t take it with me, so I might as well enjoy giving it away.”

In reporting this column, I heard from people who struggled to figure out what to give to wealthier siblings or argued with their spouses over how much money to spend. One husband says his wife likes to “spam” everyone with expensive presents, including their daughter’s new boyfriend, “who’s been in the family for five minutes.” A woman admitted that she’s still fuming that the teenage nieces she sees once or twice a year emailed her long wish lists. She pretended she was sick and skipped her family’s Christmas gathering.

Many complained about bad gifts: A newlywed wife received an electric can opener from her husband. Another got a set of dishtowels with mice embroidered on them from her new mother-in-law. A young man received deodorant balls to put in his gym shoes. “The last time I went to the gym was never,” he said.

Bad gifts burn because they make you feel unseen or even mischaracterized, says Julian Givi, an associate professor of marketing at West Virginia University, who studies gift giving. “You wonder, ‘Is this person lazy? Do they even care about me?’”

What to do

Need shopping help? Here’s some advice.

Don’t overthink it. Recipients like our gifts more than we think they do, research shows. Even when they don’t, they’re not as upset as we fear. Boring is even OK, as long as they think it’s something they’ll find useful, Polman says.

Ask them what they want. Yes, it can feel transactional to give a gift someone has asked for. But recipients prefer a gift they want, studies show.

Consider giving an experience, such as a gift certificate to your favorite restaurant. Recipients prefer this, and if the experience is something you both enjoy, it helps you bond, Liu says.

Give yourself enough time. “You’re not going to do a good job if you shop at the last minute,” Givi says.

Amanda Latimer loves a big Christmas. Last year, the 48-year-old mom of six bought her kids, aged 16 to 27, what they wanted: designer clothes and tech gear. Then she threw in extras, such as cozy blankets, to have more under the tree.

Yet on Christmas morning, Latimer says she “felt a little empty inside” after the presents were opened. Her children were appreciative. But they didn’t seem wowed.

This year, Latimer has a new plan: In lieu of presents, her family will make a donation to the Salvation Army to fulfill a needy family’s Christmas wish list. Latimer says she was inspired by her grandmother, Joan Kroc, wife of McDonald’s founder Ray Kroc, who gave the charity $1.5 billion after she died in 2003.

Her kids are excited. “I feel bad asking for anything for Christmas,” says Lauren Latimer, 19. “This feels much better.”

Write to Elizabeth Bernstein at Elizabeth.Bernstein@wsj.com