Has AI research hit a plateau in the language model (LLM) route, and do we require a new breakthrough?

LLMs have been the discussion point in AI for the past year and more. Despite significant advancements in LLMs, there's growing concern that current methods are approaching their limits. This raises the question of whether a paradigm shift is necessary for further progress. Innovations in AI architecture, data processing, or a completely new approach might be needed to surpass existing barriers. If so, what might this breakthrough entail?