Stocks finished 2023 near all-time highs with the S&P 500 (^GSPC) gaining nearly 24% on the year.

The three major averages ripped higher in the final two months of 2023 after a pivot from the Federal Reserve has many investors increasingly betting that the central bank's next interest rate adjustment will bring rates lower.

But despite the market's newfound optimism to end the year, Wall Street doesn't see much upside for stocks in 2024.

Given the rally, many strategists' S&P 500 calls for 2024 already reflect a limited increase for stocks next year. The median target among the 20 Wall Street strategists tracked by Bloomberg shows the benchmark index finishing 2024 at 4,850, less than 2% higher than where the benchmark closed 2023.

Strategists at Goldman Sachs already boosted their 2024 target to reflect the recent run-up in stocks and the shift to a more dovish Fed. Goldman boosted its S&P 500 projection from 4,700 to 5,100 on Dec. 18.

And the range for next year's targets is wide. Oppenheimer and Fundstrat are the most bullish, with year-end targets of 5,200 for the S&P 500, reflecting about 9% upside from the 2023 close. Meanwhile, the lowest call on the Street for 2024 is JPMorgan's prediction that the S&P will sink to 4,200, which would mark a 12% decline for the benchmark index in 2024.

Will recession hit and bring down stocks ?

Much of the divide between bulls and bears headed into 2024 rests on where different firms see the economy headed next year.

Those that either see the economy not entering a recession at all, or believe that outcome has been talked about so much it won't entail much impact for stocks, predict the S&P 500 hits at least 5,000 in 2024. That camp includes firms like Oppenheimer, Fundstrat, Goldman Sachs, Deutsche Bank, and Bank of America.

Brian Belski at BMO calls any pending recession the "Chicken Little recession," a reference to the fictional character who insists the sky is falling and causes mass hysteria over it. Belski thinks if there is a downturn next year it will be a "recession in name only."

"We will continue to take our cue from labor market trends, and unless they take a sharp turn for the worse, we are simply not concerned about the recession debate at this point," Belski wrote in his 2024 outlook.

The team at Deutsche Bank is still in the recession camp, though. The analysts see economic growth slowing and "a mild recession" in the first half of the year. But to the firm's chief US equity strategist Binky Chadha, the risks of recession would only lead to a "modest short-lived sell-off."

Others still see a recession weighing on stocks in 2024. Evercore ISI's Julian Emanuel wrote that stocks will be "down first into recession, then higher as inflation hits the [Fed's 2%] target." Emanuel believes the recession will come in the first half of the year before a rally leads the S&P 500 to his 4,750 target.

JPMorgan's equity strategists are even more cautious about what a downturn could spell for stocks as they project the benchmark average closing 2024 at 4,200.

"Absent rapid Fed easing, we expect a more challenging macro backdrop for stocks next year with softening consumer trends at a time when investor positioning and sentiment have mostly reversed," JPMorgan equity strategists led by Dubravko Lakos-Bujas wrote in the team's 2024 outlook on Nov. 29.

Lakos-Bujas's point about Fed easing is a key sticking point in the bulls-versus-bears argument. At a high level, there are two basic reasons the Fed would cut interest rates, which it currently forecasts it will do three times in 2024. The Fed would lower rates if the economy meaningfully slowed to ease financial conditions and help keep it afloat.

Or the Fed would cut rates because inflation falls toward the central bank's 2% target more quickly than anticipated. This is the scenario Goldman Sachs cited when boosting its outlook for stocks in mid-December.

"Resilient growth and falling rates should benefit stocks with weaker balance sheets, particularly those that are sensitive to economic growth," Goldman Sachs chief US equity strategist David Kostin wrote in a strategy note.

In the past, whether or not a recession lies ahead has played a key role in whether stocks rally or fall following the first interest rate cut. A graph from Goldman Sachs shows that stocks typically fall if a recession hits in the 12 months following the first Fed rate cut.

Will it be all about the Magnificent Seven again?

A well-documented aspect of the 2023 stock market rally was how seven large technology stocks — Apple (AAPL), Alphabet (GOOGL, GOOG), Microsoft (MSFT), Amazon (AMZN), Meta (META), Tesla (TSLA), and Nvidia (NVDA) — drove most of the market's gains. But in the final two months of the year, the rally broadened out, and many strategists see that market breadth continuing in 2024.

"We forecast an all-time high for the S&P 500 in 2024, with a year-end target of 5000. But unlike this year during which the Magnificent 7 did 70% of the work, we expect broader leadership," Bank of America head of US equity and quantitative strategy Savita Subramanian wrote in a December note to clients.

Fundstrat founder Tom Lee places Technology and FAANG stocks among his top three sectors for 2024. But after a massive run-up in 2023, Lee doesn't see tech leading the way again next year.

"Do I think there's enough juice in FAANG from earnings plus multiple expansion to outperform small caps? I don't think so," Lee said during his 2024 outlook Zoom call on Dec. 7. "I think small caps could rise 50% next year easily. And Financials could rise 30% ... When it comes to positioning no one owns Financials and no one's really long small caps. There's a lot of upside."

Kostin at Goldman Sachs also gave a shout-out to small caps in his recent 2024 outlook.

"An environment of falling interest rates and improving economic growth expectations historically has been supportive for small-caps, which have recently traded at depressed valuations," Kostin wrote.

"This likely opens the door for increased participation with traditional growth areas (especially within Technology) ...Given the outperformance of growth, we believe investors should be much more prudent and focus on themes (not just liquidity or momentum), stable growth, and even dividends within Growth sectors."

"We believe there is a very good chance that the 'Magnificent 7' will not be as unified in terms of performance trends in 2024," Belski wrote in his 2024 outlook. "For instance, company-specific fundamentals are very different, with recent price performance trends in the 4Q portending to increasingly varied performance in 2024."

"This likely opens the door for increased participation with traditional growth areas (especially within Technology) ... Given the outperformance of growth, we believe investors should be much more prudent and focus on themes (not just liquidity or momentum), stable growth, and even dividends within Growth sectors."

Josh Schafer is a reporter for Yahoo Finance.

Click here for the latest technology news that will impact the stock market.

Read the latest financial and business news from Yahoo Finance