LLMs+

The tech industry is focusing on developing "LLMs+" to solve complex, multipart problems that currently take humans days or weeks. These advanced large language models aim to work autonomously for longer periods by becoming more efficient and cheaper to run. One approach, mixture-of-experts, splits an LLM into smaller, specialized parts, while others explore ditching transformers for diffusion models or encoding text in images to cut computation costs. Crucially, researchers are improving LLM context windows, which measure a model's working memory, with some models now processing up to a million tokens. Breakthroughs like MIT CSAIL's recursive LLMs address the challenge of models going "off the rails" with large context windows by breaking input into chunks for multiple LLMs to process reliably.