How it works:\n- Agents run in Playwright-controlled browsers inside Docker sandboxes\n- Each turn, agents receive the accessibility tree + URL and return a tool call (navigate, click, type, etc.)\n- Glicko-2 ratings across 6 domains (browser tasks, prediction markets, trading, games, creative, coding)\n- Submit via webhook (5-min setup) or paste an API key
The two-way submission design lets any framework or model compete. Sandbox mode is free, no credit card required.
Code: https://github.com/stefanogebara/ai-olympics
Curious what the community thinks about the task design and whether anyone wants to test their agents against it."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"Show HN: AI Olympics \u2013 Claude vs. GPT-4 vs. Gemini in live browser competitions"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://ai-olympics.vercel.app"}},"_tags":["story","author_stefanogebara","story_47145595","show_hn"],"author":"stefanogebara","children":[47145657,47145844],"created_at":"2026-02-25T00:25:12Z","created_at_i":1771979112,"num_comments":1,"objectID":"47145595","points":2,"story_id":47145595,"story_text":"I built a platform where AI agents compete against each other in real-world internet tasks: filling out forms, extracting data, trading prediction markets, playing games, and writing code \u2014 with real-time spectating and AI commentary.
How it works:\n- Agents run in Playwright-controlled browsers inside Docker sandboxes\n- Each turn, agents receive the accessibility tree + URL and return a tool call (navigate, click, type, etc.)\n- Glicko-2 ratings across 6 domains (browser tasks, prediction markets, trading, games, creative, coding)\n- Submit via webhook (5-min setup) or paste an API key
The two-way submission design lets any framework or model compete. Sandbox mode is free, no credit card required.
Code: https://github.com/stefanogebara/ai-olympics
Curious what the community thinks about the task design and whether anyone wants to test their agents against it.","title":"Show HN: AI Olympics \u2013 Claude vs. GPT-4 vs. Gemini in live browser competitions","updated_at":"2026-03-05T23:36:47Z","url":"https://ai-olympics.vercel.app"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"stanleycyang"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"Hey HN,
I built yardstiq because I got tired of the copy-paste workflow for comparing LLM responses when developing apps. Every time I wanted to see how Claude vs GPT vs Gemini handled the same prompt, I'd open three tabs, paste the same thing, and try to eyeball the differences. It's 2026 and we have 40+ models worth considering \u2014 that doesn't scale.
yardstiq is a CLI tool that sends one prompt to multiple models simultaneously and streams the responses side-by-side in your terminal. It also tracks performance metrics (time to first token, tokens/sec, cost) and optionally runs an AI judge to score the outputs.
```\nnpx yardstiq "Explain quicksort in 3 sentences" -m claude-sonnet -m gpt-4o\n```
What it does:
- Streams responses from multiple models in parallel, rendered in columns\n- Shows TTFT, throughput (tok/s), token counts, and cost per request\n- AI judge mode: have a model evaluate and score the responses\n- Export to JSON, Markdown, or self-contained HTML reports\n- Run YAML-defined benchmark suites across models with aggregate scoring\n- Works with Ollama for local model comparisons (zero API cost)\n- Supports 40+ models via direct provider keys or Vercel AI Gateway
I built this mostly for my own workflow \u2014 picking models for different tasks, testing prompt variations, and running quick benchmarks without setting up a whole evaluation framework. It's not trying to replace serious eval platforms, just make the "which model is better for X?" question answerable in 10 seconds.
MIT licensed, written in TypeScript: https://github.com/stanleycyang/yardstiq
Happy to answer questions about the architecture or benchmarking approach."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Yardstiq \u2013 Compare LLM outputs side-by-side in your terminal"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.yardstiq.sh"}},"_tags":["story","author_stanleycyang","story_47235222","show_hn"],"author":"stanleycyang","children":[47236630,47246254,47265845],"created_at":"2026-03-03T16:54:46Z","created_at_i":1772556886,"num_comments":0,"objectID":"47235222","points":2,"story_id":47235222,"story_text":"Hey HN,
I built yardstiq because I got tired of the copy-paste workflow for comparing LLM responses when developing apps. Every time I wanted to see how Claude vs GPT vs Gemini handled the same prompt, I'd open three tabs, paste the same thing, and try to eyeball the differences. It's 2026 and we have 40+ models worth considering \u2014 that doesn't scale.
yardstiq is a CLI tool that sends one prompt to multiple models simultaneously and streams the responses side-by-side in your terminal. It also tracks performance metrics (time to first token, tokens/sec, cost) and optionally runs an AI judge to score the outputs.
```\nnpx yardstiq "Explain quicksort in 3 sentences" -m claude-sonnet -m gpt-4o\n```
What it does:
- Streams responses from multiple models in parallel, rendered in columns\n- Shows TTFT, throughput (tok/s), token counts, and cost per request\n- AI judge mode: have a model evaluate and score the responses\n- Export to JSON, Markdown, or self-contained HTML reports\n- Run YAML-defined benchmark suites across models with aggregate scoring\n- Works with Ollama for local model comparisons (zero API cost)\n- Supports 40+ models via direct provider keys or Vercel AI Gateway
I built this mostly for my own workflow \u2014 picking models for different tasks, testing prompt variations, and running quick benchmarks without setting up a whole evaluation framework. It's not trying to replace serious eval platforms, just make the "which model is better for X?" question answerable in 10 seconds.
MIT licensed, written in TypeScript: https://github.com/stanleycyang/yardstiq
Happy to answer questions about the architecture or benchmarking approach.","title":"Show HN: Yardstiq \u2013 Compare LLM outputs side-by-side in your terminal","updated_at":"2026-03-08T20:27:38Z","url":"https://www.yardstiq.sh"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"charliebwrites"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"Not sure if this is even possible, but is there any site or benchmark for testing which AI model is best for the task of coding?
Like Claude 3.5 vs GPT 4o vs Gemini 2 etc
What exists beyond our opinions to more objectively measure the quality of code output on these models?"},"title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Are there any objective measurements for AI model coding performance?"}},"_tags":["story","author_charliebwrites","story_43206779","ask_hn"],"author":"charliebwrites","children":[43207550,43210521],"created_at":"2025-02-28T15:38:03Z","created_at_i":1740757083,"num_comments":2,"objectID":"43206779","points":2,"story_id":43206779,"story_text":"Not sure if this is even possible, but is there any site or benchmark for testing which AI model is best for the task of coding?
Like Claude 3.5 vs GPT 4o vs Gemini 2 etc
What exists beyond our opinions to more objectively measure the quality of code output on these models?","title":"Ask HN: Are there any objective measurements for AI model coding performance?","updated_at":"2025-02-28T21:43:54Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Sajarin"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"I wonder if anyone has done an analysis on the HN user sentiment on the varying AI models over time. I'd be curious to see what that looks like. Increasingly, I'm seeing more and more people talk positively about Gemini and Google (and having used Gemini recently, I align with that sentiment)
I think Bard (lol) and Gemini got a late start and so lots of folks dismissed it but I feel like they've fully caught up. Definitely excited to see what Gemini 3 vs GPT-5 vs Claude 4 looks like!"},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["claude"],"value":"Claude Code SDK"},"story_url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["claude"],"value":"https://docs.anthropic.com/en/docs/claude-code/sdk"}},"_tags":["comment","author_Sajarin","story_44032777"],"author":"Sajarin","children":[44033565,44033883,44034058],"comment_text":"I wonder if anyone has done an analysis on the HN user sentiment on the varying AI models over time. I'd be curious to see what that looks like. Increasingly, I'm seeing more and more people talk positively about Gemini and Google (and having used Gemini recently, I align with that sentiment)
I think Bard (lol) and Gemini got a late start and so lots of folks dismissed it but I feel like they've fully caught up. Definitely excited to see what Gemini 3 vs GPT-5 vs Claude 4 looks like!","created_at":"2025-05-19T18:46:18Z","created_at_i":1747680378,"objectID":"44033321","parent_id":44033001,"story_id":44032777,"story_title":"Claude Code SDK","story_url":"https://docs.anthropic.com/en/docs/claude-code/sdk","updated_at":"2025-05-20T03:40:34Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"fzysingularity"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"VLM Showdown: GPT vs. Gemini vs. Claude vs. Orion"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://chat.vlm.run/showdown"}},"_tags":["story","author_fzysingularity","story_45996392"],"author":"fzysingularity","children":[45996393],"created_at":"2025-11-20T19:08:45Z","created_at_i":1763665725,"num_comments":1,"objectID":"45996392","points":15,"story_id":45996392,"title":"VLM Showdown: GPT vs. Gemini vs. Claude vs. Orion","updated_at":"2026-03-18T17:43:45Z","url":"https://chat.vlm.run/showdown"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"spobin"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"Pelican on a Bicycle: Claude Fable 5 vs. GPT-5.5 Pro vs. Gemini 3.1 Pro"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.promptfrenzy.com/showdown/svg-pelican"}},"_tags":["story","author_spobin","story_48474355"],"author":"spobin","created_at":"2026-06-10T10:47:03Z","created_at_i":1781088423,"num_comments":0,"objectID":"48474355","points":1,"story_id":48474355,"title":"Pelican on a Bicycle: Claude Fable 5 vs. GPT-5.5 Pro vs. Gemini 3.1 Pro","updated_at":"2026-06-10T10:48:23Z","url":"https://www.promptfrenzy.com/showdown/svg-pelican"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"chieund"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"Hi HN!
I built LLMKit after getting frustrated with choosing the right LLM for different projects. Instead of guessing or relying on benchmarks that don't match real use cases, I wanted to see actual performance with my own prompts.
What it does:\n\u2022 Compare up to 5 models simultaneously (GPT-4, Claude, Gemini, etc.)\n\u2022 Real-time streaming comparison - watch models race to respond\n\u2022 Custom scoring weights based on your priorities (speed vs cost vs quality)\n\u2022 System prompt support for production-realistic testing\n\u2022 TTFT (Time to First Token) metrics for latency-sensitive apps\n\u2022 No signup required, API keys stay in your browser
The "aha moment" was adding streaming comparison - seeing GPT-4 start fast but Claude catch up, or watching cost-effective models perform surprisingly well. It's like A/B testing but for LLMs.
Built with Next.js + TypeScript. The streaming implementation was tricky - had to handle different provider formats (OpenAI vs Anthropic) and parallel SSE connections."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: LLMKit \u2013 Compare LLMs side-by-side with real-time streaming"},"url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["claude","vs","gpt","vs"],"value":"https://www.llmkit.cc/model-comparison/gpt-4-vs-claude-3-5-sonnet"}},"_tags":["story","author_chieund","story_46356806","show_hn"],"author":"chieund","created_at":"2025-12-22T18:11:05Z","created_at_i":1766427065,"num_comments":0,"objectID":"46356806","points":2,"story_id":46356806,"story_text":"Hi HN!
I built LLMKit after getting frustrated with choosing the right LLM for different projects. Instead of guessing or relying on benchmarks that don't match real use cases, I wanted to see actual performance with my own prompts.
What it does:\n\u2022 Compare up to 5 models simultaneously (GPT-4, Claude, Gemini, etc.)\n\u2022 Real-time streaming comparison - watch models race to respond\n\u2022 Custom scoring weights based on your priorities (speed vs cost vs quality)\n\u2022 System prompt support for production-realistic testing\n\u2022 TTFT (Time to First Token) metrics for latency-sensitive apps\n\u2022 No signup required, API keys stay in your browser
The "aha moment" was adding streaming comparison - seeing GPT-4 start fast but Claude catch up, or watching cost-effective models perform surprisingly well. It's like A/B testing but for LLMs.
Built with Next.js + TypeScript. The streaming implementation was tricky - had to handle different provider formats (OpenAI vs Anthropic) and parallel SSE connections.","title":"Show HN: LLMKit \u2013 Compare LLMs side-by-side with real-time streaming","updated_at":"2026-03-05T23:16:26Z","url":"https://www.llmkit.cc/model-comparison/gpt-4-vs-claude-3-5-sonnet"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"slacktivism123"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":">Just not the GPT-5 series! My experiments so far put Gemini 2.5 at the top of the pack, to the point where I'd almost trust it for some tasks
Got it. The non-experts are holding it wrong!
The laymen are told "just use the app" or "just use the website". No need to worry about API keys or routers or wrapper scripts that way!
Sure.
Yet the laymen are expected to maintain a mental model of the failure modes and intended applications of Grok 4 vs Grok 4 Fast vs Gemini 2.5 Pro vs GPT-4.1 Mini vs GPT-5 vs Claude Sonnet 4.5...
It's a moving target. The laymen read the marketing puffery around each new model release and think the newest model is even more capable.
"This model sounds awesome. OpenAI does it again! Surely it can OCR my invoice PDFs this time!"
I mean, look at it:
GPT\u20115 not only outperforms previous models on benchmarks and answers questions more quickly, but\u2014most importantly\u2014is more useful for real-world queries.\n\n GPT\u20115 is our best model yet for health-related questions, empowering users to be informed about and advocate for their health. The model scores significantly higher than any previous model on HealthBench , an evaluation we published earlier this year based on realistic scenarios and physician-defined criteria.\n\n GPT\u20115 is much smarter across the board, as reflected by its performance on academic and human-evaluated benchmarks, particularly in math, coding, visual perception, and health. It sets a new state of the art across math (94.6% on AIME 2025 without tools), real-world coding (74.9% on SWE-bench Verified, 88% on Aider Polyglot), multimodal understanding (84.2% on MMMU), and health (46.2% on HealthBench Hard)\n\n The model excels across a range of multimodal benchmarks, spanning visual, video-based, spatial, and scientific reasoning. Stronger multimodal performance means ChatGPT can reason more accurately over images and other non-text inputs\u2014whether that\u2019s interpreting a chart, summarizing a photo of a presentation, or answering questions about a diagram.\nGot it. The non-experts are holding it wrong!
The laymen are told "just use the app" or "just use the website". No need to worry about API keys or routers or wrapper scripts that way!
Sure.
Yet the laymen are expected to maintain a mental model of the failure modes and intended applications of Grok 4 vs Grok 4 Fast vs Gemini 2.5 Pro vs GPT-4.1 Mini vs GPT-5 vs Claude Sonnet 4.5...
It's a moving target. The laymen read the marketing puffery around each new model release and think the newest model is even more capable.
"This model sounds awesome. OpenAI does it again! Surely it can OCR my invoice PDFs this time!"
I mean, look at it:
GPT\u20115 not only outperforms previous models on benchmarks and answers questions more quickly, but\u2014most importantly\u2014is more useful for real-world queries.\n\n GPT\u20115 is our best model yet for health-related questions, empowering users to be informed about and advocate for their health. The model scores significantly higher than any previous model on HealthBench , an evaluation we published earlier this year based on realistic scenarios and physician-defined criteria.\n\n GPT\u20115 is much smarter across the board, as reflected by its performance on academic and human-evaluated benchmarks, particularly in math, coding, visual perception, and health. It sets a new state of the art across math (94.6% on AIME 2025 without tools), real-world coding (74.9% on SWE-bench Verified, 88% on Aider Polyglot), multimodal understanding (84.2% on MMMU), and health (46.2% on HealthBench Hard)\n\n The model excels across a range of multimodal benchmarks, spanning visual, video-based, spatial, and scientific reasoning. Stronger multimodal performance means ChatGPT can reason more accurately over images and other non-text inputs\u2014whether that\u2019s interpreting a chart, summarizing a photo of a presentation, or answering questions about a diagram.\nI mean, I get the rationale Company vs. Product, but most people know the product. As in "I used ChatGPT". But if you ask who OpenAI is, they'll have no clue.
ChatGPT is in someways nicer... because their models are GPT-5.3, GPT-5.4, etc...
But when you're trying to explain that the Anthropic models are called "Opus" or "Sonnet" or "Haiku" or "Fable", but you use them in "Claude", it gets confusing quickly."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"OpenAI loses trademark dispute at EU court"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://dpa-international.com/economics/urn:newsml:dpa.com:20090101:260715-930-389143/"}},"_tags":["comment","author_mbreese","story_48921461"],"author":"mbreese","children":[48935695],"comment_text":"The linguistic gymnastics required when talking about OpenAI vs ChatGPT and Anthropic vs Claude is difficult when you're giving talk about them. At least Google vs Gemini is a little clearer.
I mean, I get the rationale Company vs. Product, but most people know the product. As in "I used ChatGPT". But if you ask who OpenAI is, they'll have no clue.
ChatGPT is in someways nicer... because their models are GPT-5.3, GPT-5.4, etc...
But when you're trying to explain that the Anthropic models are called "Opus" or "Sonnet" or "Haiku" or "Fable", but you use them in "Claude", it gets confusing quickly.","created_at":"2026-07-15T18:00:21Z","created_at_i":1784138421,"objectID":"48924740","parent_id":48923080,"story_id":48921461,"story_title":"OpenAI loses trademark dispute at EU court","story_url":"https://dpa-international.com/economics/urn:newsml:dpa.com:20090101:260715-930-389143/","updated_at":"2026-07-16T15:14:11Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"curioussquirrel"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"Disclosure: I work at RWS/TrainAI, we did this study. Recently I alluded to it in a comment and was encouraged to share it, so here it is!\nWe focus on multilingual proficiency, which tends to be understudied: most benchmarks are English-heavy or even English-only and don't tell you much about how models actually perform across languages.\nThis is our second iteration of the study. 120 linguists, 8 models, 8 languages, 4 tasks, every output blind-reviewed by 3 native speakers.
Some notable insights:
- GPT-5 is strong at text normalization and translation but regressed on content generation vs GPT-4o. Chinese outputs had spacing/punctuation issues, Polish read like "translationese" even with no source text.
- Gemini 2.5 Pro scored 4.56/5 on Kinyarwanda. In our first study (late 2024), no model could produce coherent text in that language.
- Top LLMs outscored humans working under realistic constraints (time-limited, single pass, no QA). Humans didn't rank 1st in any language. (We're now planning a follow-up to zoom in on that.)
- Tokenizer efficiency matters again: reasoning models burn 5-10x more tokens thinking. Claude Sonnet 4.5 encodes Tamil at 1.19 chars/token vs Gemini's 4.24 \u2014 ~3.5x cost difference for the same output. There has been a lot of talk about the Opus 4.7 tokenizer, this is the same issue, just in multilingual setting.
If you find the study useful and want to help us convince the execs to keep funding this, a signup on the landing page goes a long way: https://www.rws.com/artificial-intelligence/train-ai-data-se...
Happy to answer questions!"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"How well do LLMs work outside English? We tested 8 models in 8 languages [pdf]"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://info.rws.com/hubfs/2026/trainai/llm-data-gen-study-2.0-campaign/trainai-multilingual-llm-synthetic-data-gen-study-2.0.pdf"}},"_tags":["comment","author_curioussquirrel","story_47847282"],"author":"curioussquirrel","children":[47847365],"comment_text":"Disclosure: I work at RWS/TrainAI, we did this study. Recently I alluded to it in a comment and was encouraged to share it, so here it is!\nWe focus on multilingual proficiency, which tends to be understudied: most benchmarks are English-heavy or even English-only and don't tell you much about how models actually perform across languages.\nThis is our second iteration of the study. 120 linguists, 8 models, 8 languages, 4 tasks, every output blind-reviewed by 3 native speakers.
Some notable insights:
- GPT-5 is strong at text normalization and translation but regressed on content generation vs GPT-4o. Chinese outputs had spacing/punctuation issues, Polish read like "translationese" even with no source text.
- Gemini 2.5 Pro scored 4.56/5 on Kinyarwanda. In our first study (late 2024), no model could produce coherent text in that language.
- Top LLMs outscored humans working under realistic constraints (time-limited, single pass, no QA). Humans didn't rank 1st in any language. (We're now planning a follow-up to zoom in on that.)
- Tokenizer efficiency matters again: reasoning models burn 5-10x more tokens thinking. Claude Sonnet 4.5 encodes Tamil at 1.19 chars/token vs Gemini's 4.24 \u2014 ~3.5x cost difference for the same output. There has been a lot of talk about the Opus 4.7 tokenizer, this is the same issue, just in multilingual setting.
If you find the study useful and want to help us convince the execs to keep funding this, a signup on the landing page goes a long way: https://www.rws.com/artificial-intelligence/train-ai-data-se...
Happy to answer questions!","created_at":"2026-04-21T11:24:24Z","created_at_i":1776770664,"objectID":"47847283","parent_id":47847282,"story_id":47847282,"story_title":"How well do LLMs work outside English? We tested 8 models in 8 languages [pdf]","story_url":"https://info.rws.com/hubfs/2026/trainai/llm-data-gen-study-2.0-campaign/trainai-multilingual-llm-synthetic-data-gen-study-2.0.pdf","updated_at":"2026-04-22T03:39:51Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mythz"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"Gemini CLI is too slow to be useful, kind of surprised it was even offered and marketed given how painful it is to use. I thought it'd have to be damaging to the Gemini brand to get people to try it out, suffer painful UX then immediately stop using it. (Using it from Australia may also contribute to its slow perf)
Antigravity was also painful to use at launch where more queries failed then succeeded, however they've basically solved that now to the point where it's become my most used editor/IDE where I've yet to hit a quota limit, despite only being on the $20/mo plan - even when using Gemini 3 Pro as the default model. I also can't recall seeing any failed service responses after a month of full-time usage. It's not the fastest model, but very happy with its high quality output.
I expected to upgrade to a Claude Code Max plan after leaving Augment Code, but given how good Antigravity is now for its low cost, I've switched to it as my primary full-time coding assistant.
Still paying for GitHub Copilot / Claude Pro for general VS Code and CC terminal usage, but definitely getting the most value of out my Gemini AI Pro sub.
Note this is only for development, docs and other work product. For API usage in products, I primarily lean on the cheaper OSS chinese models, primarily MiniMax 2.1 for tool calling or GLM 4.7/KimiK2/DeepSeek when extra intelligence is needed (at slower perf). Gemini Flash for analyzing Image, Audio & PDFs.
Also find Nano Banana/Pro (Gemini Flash Image) to consistently generate the highest quality images vs GPT 1.5/SDXL,HiDream,Flux,ZImage,Qwen, which apparently my Pro sub includes up to 1000/day for Nano Banana or 100/day for Pro?? [1], so it's hard to justify using anything else.
If Gemini 3 Pro was a bit faster and Flash a bit cheaper (API Usage), I could easily see myself switching to Gemini for everything. If future releases get smarter, faster whilst remaining aggressively priced, in the future - I expect I will.
[1] https://support.google.com/gemini/answer/16275805?hl=en"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"How Google got its groove back and edged ahead of OpenAI"},"story_url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["gemini"],"value":"https://www.wsj.com/tech/ai/google-ai-openai-gemini-chatgpt-b766e160"}},"_tags":["comment","author_mythz","story_46528389"],"author":"mythz","children":[46537859,46539836,46540147],"comment_text":"Gemini CLI is too slow to be useful, kind of surprised it was even offered and marketed given how painful it is to use. I thought it'd have to be damaging to the Gemini brand to get people to try it out, suffer painful UX then immediately stop using it. (Using it from Australia may also contribute to its slow perf)
Antigravity was also painful to use at launch where more queries failed then succeeded, however they've basically solved that now to the point where it's become my most used editor/IDE where I've yet to hit a quota limit, despite only being on the $20/mo plan - even when using Gemini 3 Pro as the default model. I also can't recall seeing any failed service responses after a month of full-time usage. It's not the fastest model, but very happy with its high quality output.
I expected to upgrade to a Claude Code Max plan after leaving Augment Code, but given how good Antigravity is now for its low cost, I've switched to it as my primary full-time coding assistant.
Still paying for GitHub Copilot / Claude Pro for general VS Code and CC terminal usage, but definitely getting the most value of out my Gemini AI Pro sub.
Note this is only for development, docs and other work product. For API usage in products, I primarily lean on the cheaper OSS chinese models, primarily MiniMax 2.1 for tool calling or GLM 4.7/KimiK2/DeepSeek when extra intelligence is needed (at slower perf). Gemini Flash for analyzing Image, Audio & PDFs.
Also find Nano Banana/Pro (Gemini Flash Image) to consistently generate the highest quality images vs GPT 1.5/SDXL,HiDream,Flux,ZImage,Qwen, which apparently my Pro sub includes up to 1000/day for Nano Banana or 100/day for Pro?? [1], so it's hard to justify using anything else.
If Gemini 3 Pro was a bit faster and Flash a bit cheaper (API Usage), I could easily see myself switching to Gemini for everything. If future releases get smarter, faster whilst remaining aggressively priced, in the future - I expect I will.
[1] https://support.google.com/gemini/answer/16275805?hl=en","created_at":"2026-01-08T06:12:34Z","created_at_i":1767852754,"objectID":"46537782","parent_id":46528389,"story_id":46528389,"story_title":"How Google got its groove back and edged ahead of OpenAI","story_url":"https://www.wsj.com/tech/ai/google-ai-openai-gemini-chatgpt-b766e160","updated_at":"2026-03-05T23:18:57Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"OJFord"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"Also people don't talk enough about (or are bad at separating themselves) the model vs. the client tool - e.g. from your comment maybe using codex/Claude Code/aider with Gemini API would be better, best even, but people rarely make that comparison or separation, it's always 'Claude Code with Claude vs. codex with GPT-x' etc."},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["gemini"],"value":"Run interactive commands in Gemini CLI"},"story_url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["gemini"],"value":"https://developers.googleblog.com/en/say-hello-to-a-new-level-of-interactivity-in-gemini-cli/"}},"_tags":["comment","author_OJFord","story_45605823"],"author":"OJFord","children":[45680860,45682485],"comment_text":"Also people don't talk enough about (or are bad at separating themselves) the model vs. the client tool - e.g. from your comment maybe using codex/Claude Code/aider with Gemini API would be better, best even, but people rarely make that comparison or separation, it's always 'Claude Code with Claude vs. codex with GPT-x' etc.","created_at":"2025-10-23T09:48:57Z","created_at_i":1761212937,"objectID":"45680057","parent_id":45679870,"story_id":45605823,"story_title":"Run interactive commands in Gemini CLI","story_url":"https://developers.googleblog.com/en/say-hello-to-a-new-level-of-interactivity-in-gemini-cli/","updated_at":"2026-03-05T22:53:26Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Topfi"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"Fair point of course and it is still far to early to make a definitive statement, but in my still limited experience throughout the night, I have seen Haiku 4.5 be far better in using what I'd consider a justifiable amount of input tokens over e.g. GPT-5 models. Sonnets recent versions also had been better on this front over OpenAIs current best, but I try (not always succeed) to take prior experience and expectation out of the equation when evaluating models.
Additionally, the AA cost to run benchmark suite numbers are very encouraging [0] and Haiku 4.5 without reasoning is always an option too. Tested that even less, but there is some indication that reasoning may not be necessary for reasonable output performance [1][2][3].
In retrospect, I perhaps would have been served better starting with "reasoning" disabled, will have to do some self-blinded comparisons between model outputs over the coming weeks to rectify that. Am trying my best not to make a judgement yet, but compared to other recent releases, Haiku 4.5 has a very interesting, even distribution.
GPT-5 models were and continue to be encouraging for price/performance with a reliable 400k window and good adherence to prompts with multi minute (beyond 10) adherence, but from the start weren't the fastest and ingests every token there is in a code base with reckless abandon.
No Grok model ever performed for me like they seem to during the initial hype
GLM-4.6 is great value but still not solid enough for tool calls, not that fast, etc. so if you can afford something more reliable I'd go for that, but encouraging.
Recent Anthropic releases were good at code output quality, but not as reliable beyond 200k vs GPT-5, not exactly fast either when looking at token/sec, though task completion generally takes less time due to more efficient ingestion vs GPT-5 and of course rather expensive.
Haiku 4.5, if they can continue to offer it at such speeds with such low latency and at this price, cupeled with encouraging initial output quality and efficient ingestion of repos seems to be designed in a far more balanced manner, which I welcome. Course with 200k being a hard limit, that is a clear downside compared to GPT-5 (and Gemini 2.5 Pro though that has its own reliability issues in tool calling) and I have yet to test whether it can go beyond 8 min on chains of tool calls with intermittent code changes without suffering similar degradation to other recent Anthropic models, but I am seeing the potential for solid value here.
[0] https://artificialanalysis.ai/?models=gpt-5-codex%2Cgpt-5-mi...
[1] Claude 4.5 Haiku\n198.72 tok/sec\n2382 tokens\nTime-to-First: 1.0 sec\nhttps://t3.chat/share/35iusmgsw9
[2] Claude 4.5 Haiku\n197.51 tok/sec\n3128 tokens\nTime-to-First: 0.91 sec\nhttps://t3.chat/share/17mxerzlj1
[3] Claude 4.5 Haiku\n154.75 tok/sec\n2341 tokens\nTime-to-First: 0.50 sec\nhttps://t3.chat/share/96wfkxzsdk"},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["claude"],"value":"Claude Haiku 4.5"},"story_url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["claude"],"value":"https://www.anthropic.com/news/claude-haiku-4-5"}},"_tags":["comment","author_Topfi","story_45595403"],"author":"Topfi","children":[45609233],"comment_text":"Fair point of course and it is still far to early to make a definitive statement, but in my still limited experience throughout the night, I have seen Haiku 4.5 be far better in using what I'd consider a justifiable amount of input tokens over e.g. GPT-5 models. Sonnets recent versions also had been better on this front over OpenAIs current best, but I try (not always succeed) to take prior experience and expectation out of the equation when evaluating models.
Additionally, the AA cost to run benchmark suite numbers are very encouraging [0] and Haiku 4.5 without reasoning is always an option too. Tested that even less, but there is some indication that reasoning may not be necessary for reasonable output performance [1][2][3].
In retrospect, I perhaps would have been served better starting with "reasoning" disabled, will have to do some self-blinded comparisons between model outputs over the coming weeks to rectify that. Am trying my best not to make a judgement yet, but compared to other recent releases, Haiku 4.5 has a very interesting, even distribution.
GPT-5 models were and continue to be encouraging for price/performance with a reliable 400k window and good adherence to prompts with multi minute (beyond 10) adherence, but from the start weren't the fastest and ingests every token there is in a code base with reckless abandon.
No Grok model ever performed for me like they seem to during the initial hype
GLM-4.6 is great value but still not solid enough for tool calls, not that fast, etc. so if you can afford something more reliable I'd go for that, but encouraging.
Recent Anthropic releases were good at code output quality, but not as reliable beyond 200k vs GPT-5, not exactly fast either when looking at token/sec, though task completion generally takes less time due to more efficient ingestion vs GPT-5 and of course rather expensive.
Haiku 4.5, if they can continue to offer it at such speeds with such low latency and at this price, cupeled with encouraging initial output quality and efficient ingestion of repos seems to be designed in a far more balanced manner, which I welcome. Course with 200k being a hard limit, that is a clear downside compared to GPT-5 (and Gemini 2.5 Pro though that has its own reliability issues in tool calling) and I have yet to test whether it can go beyond 8 min on chains of tool calls with intermittent code changes without suffering similar degradation to other recent Anthropic models, but I am seeing the potential for solid value here.
[0] https://artificialanalysis.ai/?models=gpt-5-codex%2Cgpt-5-mi...
[1] Claude 4.5 Haiku\n198.72 tok/sec\n2382 tokens\nTime-to-First: 1.0 sec\nhttps://t3.chat/share/35iusmgsw9
[2] Claude 4.5 Haiku\n197.51 tok/sec\n3128 tokens\nTime-to-First: 0.91 sec\nhttps://t3.chat/share/17mxerzlj1
[3] Claude 4.5 Haiku\n154.75 tok/sec\n2341 tokens\nTime-to-First: 0.50 sec\nhttps://t3.chat/share/96wfkxzsdk","created_at":"2025-10-16T08:16:28Z","created_at_i":1760602588,"objectID":"45602757","parent_id":45596451,"story_id":45595403,"story_title":"Claude Haiku 4.5","story_url":"https://www.anthropic.com/news/claude-haiku-4-5","updated_at":"2026-03-05T22:48:02Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"shricodevvv"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"OpenAI GPT-5.2-Codex (High) vs. Claude Opus 4.5 vs. Gemini 3 Pro (In Production)"},"url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["vs","vs","gemini"],"value":"https://www.tensorlake.ai/blog/gpt5.2-codex-high-vs-opus-4.5-vs-gemini-3-pro"}},"_tags":["story","author_shricodevvv","story_46688306"],"author":"shricodevvv","created_at":"2026-01-20T05:44:07Z","created_at_i":1768887847,"num_comments":0,"objectID":"46688306","points":1,"story_id":46688306,"title":"OpenAI GPT-5.2-Codex (High) vs. Claude Opus 4.5 vs. Gemini 3 Pro (In Production)","updated_at":"2026-03-05T23:28:47Z","url":"https://www.tensorlake.ai/blog/gpt5.2-codex-high-vs-opus-4.5-vs-gemini-3-pro"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"owendarko"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"What a lot of people don\u2019t mention here is the choice of an AI model.
The key word in the OP post is \u201cClaude\u201d. Anthropic has 2 amazing AI models (Sonnet, Opus), however they\u2019re just a part of a much bigger picture.
When using AI for programming, you\u2019re essentially interacting with AI models. The quality of output you get really depends on the model at the end of the day.\nClaude Code is optimized for models from Anthropic. You also have model-agnostic agents like Cursor and Kilo Code (disclaimer: I work at Kilo) where you can easily switch up models and see which one works the best for you converting an old jQuery + Django project into SvelteKit.
This area is moving at a crazy rate. Just the last 2 weeks alone, there were 3 main AI model versions released (first Gemini 3, then Opus 4.5, then GPT-5.2). I wrote a piece comparing their performance across 3 coding tasks [1]
So my advice is to also experiment a lot with the models because the quality can vary wildly depending on your stack.
[1] https://blog.kilo.ai/p/we-tested-gpt-52pro-vs-opus-45-vs"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: How can I get better at using AI for programming?"}},"_tags":["comment","author_owendarko","story_46255285"],"author":"owendarko","comment_text":"What a lot of people don\u2019t mention here is the choice of an AI model.
The key word in the OP post is \u201cClaude\u201d. Anthropic has 2 amazing AI models (Sonnet, Opus), however they\u2019re just a part of a much bigger picture.
When using AI for programming, you\u2019re essentially interacting with AI models. The quality of output you get really depends on the model at the end of the day.\nClaude Code is optimized for models from Anthropic. You also have model-agnostic agents like Cursor and Kilo Code (disclaimer: I work at Kilo) where you can easily switch up models and see which one works the best for you converting an old jQuery + Django project into SvelteKit.
This area is moving at a crazy rate. Just the last 2 weeks alone, there were 3 main AI model versions released (first Gemini 3, then Opus 4.5, then GPT-5.2). I wrote a piece comparing their performance across 3 coding tasks [1]
So my advice is to also experiment a lot with the models because the quality can vary wildly depending on your stack.
[1] https://blog.kilo.ai/p/we-tested-gpt-52pro-vs-opus-45-vs","created_at":"2025-12-14T13:04:38Z","created_at_i":1765717478,"objectID":"46262736","parent_id":46255285,"story_id":46255285,"story_title":"Ask HN: How can I get better at using AI for programming?","updated_at":"2026-03-05T23:11:38Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tw1984"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["claude","vs","gpt","vs","gemini"],"value":"really love your dual standard mate!
according to the SWE bench results I am looking at, KIMI K2 has higher agentic coding score than Gemini and its gap with Claude Haiku 4.5 is just 71.3% vs 73.3%, that 2% difference is actually less than the 3% gap between GPT 5.1 (76.3%) vs Claude Haiku 4.5. interestingly, Gemini and Claude Haiku 4.5 are "frontier" according to you but KIMI K2, which actually has the higest HLE nd Live Codebench results, is just "near" the frontier."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Disrupting the first reported AI-orchestrated cyber espionage campaign"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.anthropic.com/news/disrupting-AI-espionage"}},"_tags":["comment","author_tw1984","story_45918638"],"author":"tw1984","children":[45936051],"comment_text":"really love your dual standard mate!
according to the SWE bench results I am looking at, KIMI K2 has higher agentic coding score than Gemini and its gap with Claude Haiku 4.5 is just 71.3% vs 73.3%, that 2% difference is actually less than the 3% gap between GPT 5.1 (76.3%) vs Claude Haiku 4.5. interestingly, Gemini and Claude Haiku 4.5 are "frontier" according to you but KIMI K2, which actually has the higest HLE nd Live Codebench results, is just "near" the frontier.","created_at":"2025-11-14T10:51:27Z","created_at_i":1763117487,"objectID":"45925704","parent_id":45924611,"story_id":45918638,"story_title":"Disrupting the first reported AI-orchestrated cyber espionage campaign","story_url":"https://www.anthropic.com/news/disrupting-AI-espionage","updated_at":"2026-03-05T22:59:09Z"}],"hitsPerPage":20,"nbHits":65,"nbPages":4,"page":0,"params":"query=Claude+vs+GPT+vs+Gemini&advancedSyntax=true&analyticsTags=backend","processingTimeMS":20,"processingTimingsMS":{"_request":{"queue":2,"roundTrip":24},"afterFetch":{"format":{"highlighting":1,"total":1},"merge":{"mergeLoop":{"total":1},"total":1},"total":1},"fetch":{"query":5,"scanning":12,"total":18},"total":20},"query":"Claude vs GPT vs Gemini","serverTimeMS":24}