LLM Context Windows Need to Expand.
Therefore, Frontier LLM Spend Will Continue.
If you pay Anthropic $20 month, the model’s context window is approximately 200K tokens, which is the equivalent of approximately 500 pages of text. The output limit is 64K tokens. If you are an Enterprise customer or use the API, you get a context window of 500K-1 million tokens.
Doubling the context window size roughly quadruples the compute. The frontier models would benefit significantly from larger context windows. Barring a new architecture or tricks like conversation compression, Anthropic, Google, xAI and OpenAI will need to invest heavily to drive LLM performance. Competition with each other and with China will ensure this happens. Google’s deep pockets and its DeepMind AI research lab give it an advantage in terms of how it may position itself to maximize Gemini’s performance.
