DeepSeek's Improved Performance
I don’t put much credence into the AI Evals given that real-world model performance is nowhere close to eval performance. Further, I believe the model companies train on the evals. That said, DeepSeek continues to improve (chart below).
How will OpenAI and Anthropic compete long-term with well-funded Google (GOOGL) Gemini on the one-hand, and inexpensive opensource models such as DeepSeek on the other?
Steady state, most language models will be opensource. China’s strategy of flooding the market with high quality opensource models is brilliant. The U.S. is taking the wrong approach by pursuing a closed model strategy. The real IP will be what people build on top of the models, not the models themselves.
