More than half, 56%, of Americans decline social invitations in order to save money, according to a survey of more than 1,200 people by credit fintech company Self Financial. And the move pays off; respondents said they save almost $1,000 per year by opting out of common social activities like dinner, movies, or happy hour.

This rings especially true today when consumers are still feeling the pinch of pandemic inflation. Restaurant food and cocktail prices are up 4.1% and 2.1%, respectively, from last year, according to recent data from the Bureau of Labor Statistics.

While saving money can alleviate some anxiety, missing out on gatherings can drum up some negative emotions. In fact, 60% of Americans say they feel jealous of other people's good times when scrolling on social media, according to the survey.

FOMO isn't the only downside of saying "no" to plans. Quality relationships are crucial factors in both happiness and longevity, research shows.

Becoming a homeowner or paying off your student loans are amazing accomplishments, but wouldn't they feel that much sweeter if you had someone with whom you could share that triumph, says Nashira Lynton, an accredited financial counselor and founder of Renewed Wealth Therapy.

"You hit the big goal, you paid off the debt — who are you celebrating with?" she says. "Know which relationships are valuable to you and which you want to maintain."

1. Decide which relationships you want to invest in

When you want to cut down on non-essential spending, you look at your budget and identify areas that you feel aren't bringing you joy.

The same goes for relationships, says Amanda Clayman, a mental health clinician who specializes in money issues. Look at how much money you're putting into each person with whom you spend time. Does it match how much you value their friendship?

"When you're reviewing your social spending, really luxuriate in the things that felt good," she says. "Who do you really like to spend time with? Are certain people not great for you or do you tend to overspend with them? The review process is helpful in spotting these differences."

Let's say you're going to expensive dinners once a month with a friend who makes you anxious. Perhaps it's worth cutting down how much time, and as result how much money, you're spending to maintain that relationship.

2. Tell people you're trying to save money

When you decline an invite but don't divulge a reason, people jump to conclusions. Letting your friends know that you're working toward a lofty savings goal or simply trying to cut down on spending gives them a clear and reasonable excuse, Lynton says.

It also helps to figure out what your boundaries are before you say "no," so you're able to effectively express them.

"When you're communicating with friends and family just be open about what you're willing to do," Lynton says. "Maybe you're willing to attend the social event but don't want to go to dinner beforehand."

If you don't know your boundaries, it's easier to get talked into spending more than you want.

3. Plan ahead

"Try to populate your calendar with social events that are low-cost," Clayman says. "Going for a walk-and-talk or making dinner at one of your houses. Put in some kind of inexpensive social support so you're not leaving yourself in a social deficit."

Bernadette Joy, a self-made millionaire who paid off $300,000 of debt, cut down on social spending by hosting game nights.

"Invite friends over and break out the board games, card games, or even video games," she says. "It's a low-cost way to enjoy each other's company."

Joy, who now runs her own money coaching company, says that even though she's happy to not be financially strained any more, many of her most cherished memories were birthed from the necessity to cut spending.

"Some of our happiest times were on that journey to paying off our debt and we had to get creative with how we spent time with people," she says.

4. Set aside a 'spontaneous yes fund'

It can be easy and even relieving to cut out fussy friends, but the reality is there will be plans that you really want to go to but just can't afford.

"The thing that hurts a lot is when someone asks us to do something and we don't have the money to say yes," Clayman says.

That's why you need to set aside a "spontaneous yes fund," she says. This can be whatever dollar amount at whatever frequency feels right to you. The point is that you can accept invitations to plans that weren't on your calendar without totally unraveling your budget.

Want to stop worrying about money? Sign up for CNBC's new online course Achieve Financial Wellness: Be Happier, Wealthier & More Financially Secure. We'll teach you the psychology of money, how to manage your stress and create healthy habits, and simple ways to boost your savings, get out of debt and invest for the future. Start today and use code EARLYBIRD for an introductory discount of 30% off through September 2, 2024.

Plus, sign up for CNBC Make It's newsletter to get tips and tricks for success at work, with money and in life.