AI 精选动态
智能评分 62
OpenAI 推理成本降低超一半
AI 推荐理由
原文披露了 OpenAI 推理成本的具体降幅和 GPU 用量，对理解其竞争优势有参考价值。核心解读
The Information 报道，OpenAI 发现新的推理优化技术，使模型运行成本降低一半以上。工程师透露，一度仅用几百块 Nvidia GPU 即可为免费用户提供 ChatGPT 服务。OpenAI 一季度毛利率为 39%，计划年底达到 52%。
全文
OpenAI reportedly found new inference optimizations that more than halved the cost of running its models!
According to The Information, engineers told colleagues this month that the techniques helped power ChatGPT for visitors without free or paid accounts using only a couple hundred Nvidia GPUs at one point.
The exact method is unclear. It could involve quantization, KV caching, batching, routing simpler queries to cheaper models, or some mix of all of those.
The business angle is bigger than the technical detail: OpenAI ended Q1 with a 39% gross margin and wants to reach 52% by year-end. Lower inference costs give it room to either improve margins, raise ChatGPT usage limits, or cut API pricing pressure on developers.
OpenAI's moat is increasingly becoming inference and cost advantage, especially against Anthropic.
Chubby♨️ (@kimmonismus): https://t.co/fYAcJi680C