(Which I still think is bonkers, FWIW.)
Opus 4.6 is an especially interesting comparison point, I think, because it was a step change; IMO it's when LLMs became serviceable for coding. Yeah, pre-4.6 models did output code, and that code often superficially worked; and bringing it up to production standards still generally meant rewriting it entirely. Opus 4.6 is when that changed.
From my early tests, it's looking like the public benchmarks are not misleading, and Qwen 3.8 somehow got there too, by and large. I've got a few personal tests. One is a mid-complexity one-shot, purposefully underspecified. Beyond a few minor bugs that it could easily fix once pointed out, Qwen 3.8 largely aced it. There are a bunch of things I'd improve, but that was true of Opus 4.6's output too, and by and large the code is clean and well structured. Also worth noting that I'm running Qwen 3.8 fairly aggressively quantized to fit in VRAM; I'd expect tighter results still from the full weights.
Another test I ran is a variant of a common puzzle with an additional structural constraint that makes the usual solution inapplicable, so the model has to actively turn away from the well-known solution and construct a new one that takes the constraint into account. I've never seen a home model pass that test. Kimi K3 passes it, GLM 5.2 passes it (painstakingly). Qwen 3.8 struggles a lot... but does arrive at a correct solution. First time I see a home model do so.
I haven't yet tested it on long multi-turn scenarios. In my experience, that's where pocket models are weakest against heavyweight ones, especially when quantized.
That aside, it does seem like Qwen 3.8 can, in fact, trade blows with Opus 4.6. I don't know yet if it could replace it, and my money would be on no, but I may well be wrong about that considering how weirdly capable it is.
Interestingly, Qwen 3.8's MTP layer is uncannily accurate too. It still gave me good results up to 6 to 8 predicted tokens, which boosts its speed so much it's competitive with Qwen 3.6 MoE. So that's another bizarrely impressive thing about it.
And given all of the above, I do think that the trillions of dollars invested into OpenAI and Anthropic are becoming harder and harder to justify."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Qwen 3.8 27B"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://huggingface.co/Qwen/Qwen3.8-27B-FP8"}},"_tags":["comment","author_Balinares","story_49299605"],"author":"Balinares","comment_text":"That's a reasonable point, so let me qualify: at a glance, it seems Qwen 3.8 27B can trade blows with Opus 4.6 on coding tasks, where trading blows doesn't necessarily mean it's a clear winner or even an equal, but does mean it'll at least hold its own and land a punch or two.
(Which I still think is bonkers, FWIW.)
Opus 4.6 is an especially interesting comparison point, I think, because it was a step change; IMO it's when LLMs became serviceable for coding. Yeah, pre-4.6 models did output code, and that code often superficially worked; and bringing it up to production standards still generally meant rewriting it entirely. Opus 4.6 is when that changed.
From my early tests, it's looking like the public benchmarks are not misleading, and Qwen 3.8 somehow got there too, by and large. I've got a few personal tests. One is a mid-complexity one-shot, purposefully underspecified. Beyond a few minor bugs that it could easily fix once pointed out, Qwen 3.8 largely aced it. There are a bunch of things I'd improve, but that was true of Opus 4.6's output too, and by and large the code is clean and well structured. Also worth noting that I'm running Qwen 3.8 fairly aggressively quantized to fit in VRAM; I'd expect tighter results still from the full weights.
Another test I ran is a variant of a common puzzle with an additional structural constraint that makes the usual solution inapplicable, so the model has to actively turn away from the well-known solution and construct a new one that takes the constraint into account. I've never seen a home model pass that test. Kimi K3 passes it, GLM 5.2 passes it (painstakingly). Qwen 3.8 struggles a lot... but does arrive at a correct solution. First time I see a home model do so.
I haven't yet tested it on long multi-turn scenarios. In my experience, that's where pocket models are weakest against heavyweight ones, especially when quantized.
That aside, it does seem like Qwen 3.8 can, in fact, trade blows with Opus 4.6. I don't know yet if it could replace it, and my money would be on no, but I may well be wrong about that considering how weirdly capable it is.
Interestingly, Qwen 3.8's MTP layer is uncannily accurate too. It still gave me good results up to 6 to 8 predicted tokens, which boosts its speed so much it's competitive with Qwen 3.6 MoE. So that's another bizarrely impressive thing about it.
And given all of the above, I do think that the trillions of dollars invested into OpenAI and Anthropic are becoming harder and harder to justify.","created_at":"2026-08-16T11:35:42Z","created_at_i":1786880142,"objectID":"49319073","parent_id":49302610,"story_id":49299605,"story_title":"Qwen 3.8 27B","story_url":"https://huggingface.co/Qwen/Qwen3.8-27B-FP8","updated_at":"2026-08-16T11:47:58Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"gymbeaux"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"LLMs have taken much of the enjoyment out of coding for me, but I don\u2019t think it has to be that way. I hope that as an industry we settle on LLMs being more like tools than human assistants. I think most of the engineers at my company are using LLMs as human assistants- most of them have agentic workflows set up and have premium subscriptions to Claude AND Gemini AND ChatGPT. Many have local LLMs running on their company MacBook Pros, but they can never manage to describe to me in plain English what those local LLMs are doing. I would compare local LLMs to Raspberry Pi clusters. They\u2019re neat, and technically they can do stuff but they are incredibly impractical. I want so desperately to have the power of Claude Opus running locally, but we are incredibly, ridiculously, extremely far off (benchmarks are often misleading and nobody likes to talk about the paltry tokens/s they\u2019re getting on their home rig).
So the problem for me is that I\u2019ve noticed a trend with my coworkers- they usually don\u2019t have a formal CS background (e.g., Comp Sci degree), their resumes are unimpressive, and they have personal websites where anyone can download a copy of their resume. I haven\u2019t seen this at any previous job. It\u2019s just weird, man. Anyway, so my problem is when they are purported to be the best of the best at this company, and they\u2019re chugging the LLM kool-aid, I\u2019m struggling to figure out whether they\u2019re really ahead of the curve and getting the drop on the rest of us, or they\u2019re doing basically no work and waving through each other\u2019s PRs with little to no oversight on what the LLMs are writing. The bush league nature of many of the bugs I\u2019m seeing going all the way to prod makes me think it\u2019s the latter.
Meanwhile, these guys and their teams are completing an absurd number of story points each sprint. My team is dead last (and coincidentally we\u2019re all pretty anti-AI for writing code).
I\u2019m waiting for the other shoe to drop. I\u2019m waiting for the C suite to call our project a failure and lay everyone off. The product generally works, though I think if the devs were truly rockstars, things would be much smoother. It seems like LLMs are letting middle-of-the-road or mediocre devs appear to be rockstars purely on output speed. Nobody in this org cares if the output has 4x the bugs it should have, just that it was completed quickly.
The main issue I think is that nobody- including the rockstar tech leads- knows how the system works. They don\u2019t know the code. There\u2019s a bug and it takes forever to track down. Don\u2019t ask about unit tests.
I think in a perfect world, these LLMs are treated like another tool rather than another person- meaning you and I still write a large portion of the code, and we read and understand every line- especially the lines generated by LLMs. It\u2019s still a huge productivity multiplier for me to not have to remember how to do mundane things like write out arrays into CSV files- before I\u2019d find a close implementation on StackOverflow and tweak to my needs. Now I can just describe what I want the CSV to look like and Claude Code generally gets it right on the first try."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What's gonna happen to software engineers?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://yakko.dev/blog/whats-gonna-happen-to-software-developers"}},"_tags":["comment","author_gymbeaux","story_48364284"],"author":"gymbeaux","comment_text":"LLMs have taken much of the enjoyment out of coding for me, but I don\u2019t think it has to be that way. I hope that as an industry we settle on LLMs being more like tools than human assistants. I think most of the engineers at my company are using LLMs as human assistants- most of them have agentic workflows set up and have premium subscriptions to Claude AND Gemini AND ChatGPT. Many have local LLMs running on their company MacBook Pros, but they can never manage to describe to me in plain English what those local LLMs are doing. I would compare local LLMs to Raspberry Pi clusters. They\u2019re neat, and technically they can do stuff but they are incredibly impractical. I want so desperately to have the power of Claude Opus running locally, but we are incredibly, ridiculously, extremely far off (benchmarks are often misleading and nobody likes to talk about the paltry tokens/s they\u2019re getting on their home rig).
So the problem for me is that I\u2019ve noticed a trend with my coworkers- they usually don\u2019t have a formal CS background (e.g., Comp Sci degree), their resumes are unimpressive, and they have personal websites where anyone can download a copy of their resume. I haven\u2019t seen this at any previous job. It\u2019s just weird, man. Anyway, so my problem is when they are purported to be the best of the best at this company, and they\u2019re chugging the LLM kool-aid, I\u2019m struggling to figure out whether they\u2019re really ahead of the curve and getting the drop on the rest of us, or they\u2019re doing basically no work and waving through each other\u2019s PRs with little to no oversight on what the LLMs are writing. The bush league nature of many of the bugs I\u2019m seeing going all the way to prod makes me think it\u2019s the latter.
Meanwhile, these guys and their teams are completing an absurd number of story points each sprint. My team is dead last (and coincidentally we\u2019re all pretty anti-AI for writing code).
I\u2019m waiting for the other shoe to drop. I\u2019m waiting for the C suite to call our project a failure and lay everyone off. The product generally works, though I think if the devs were truly rockstars, things would be much smoother. It seems like LLMs are letting middle-of-the-road or mediocre devs appear to be rockstars purely on output speed. Nobody in this org cares if the output has 4x the bugs it should have, just that it was completed quickly.
The main issue I think is that nobody- including the rockstar tech leads- knows how the system works. They don\u2019t know the code. There\u2019s a bug and it takes forever to track down. Don\u2019t ask about unit tests.
I think in a perfect world, these LLMs are treated like another tool rather than another person- meaning you and I still write a large portion of the code, and we read and understand every line- especially the lines generated by LLMs. It\u2019s still a huge productivity multiplier for me to not have to remember how to do mundane things like write out arrays into CSV files- before I\u2019d find a close implementation on StackOverflow and tweak to my needs. Now I can just describe what I want the CSV to look like and Claude Code generally gets it right on the first try.","created_at":"2026-06-02T02:28:29Z","created_at_i":1780367309,"objectID":"48365239","parent_id":48364284,"story_id":48364284,"story_title":"What's gonna happen to software engineers?","story_url":"https://yakko.dev/blog/whats-gonna-happen-to-software-developers","updated_at":"2026-06-04T09:31:30Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"maeil"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"I'm building a product that requires complex LLM flows and out of OpenAI's "cheap" tier models, the old versions of Turbo-3.5 are far better than the last versions of it and 4o-mini. I have a number of tasks that the former consistently succeed at and the latter consistently fail at regardless of prompting.
Leaderboards and benchmarks are very misleading as OpenAI is optimizing for them, like in the past when certain CPU manufacturers would optimize for synthetic benchmarks.
Fwif these aren't chat usecases, for which the newer models may well be better."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Structured Outputs in the API"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://openai.com/index/introducing-structured-outputs-in-the-api/"}},"_tags":["comment","author_maeil","story_41173223"],"author":"maeil","comment_text":"I'm building a product that requires complex LLM flows and out of OpenAI's "cheap" tier models, the old versions of Turbo-3.5 are far better than the last versions of it and 4o-mini. I have a number of tasks that the former consistently succeed at and the latter consistently fail at regardless of prompting.
Leaderboards and benchmarks are very misleading as OpenAI is optimizing for them, like in the past when certain CPU manufacturers would optimize for synthetic benchmarks.
Fwif these aren't chat usecases, for which the newer models may well be better.","created_at":"2024-08-07T04:41:59Z","created_at_i":1723005719,"objectID":"41178291","parent_id":41177796,"story_id":41173223,"story_title":"Structured Outputs in the API","story_url":"https://openai.com/index/introducing-structured-outputs-in-the-api/","updated_at":"2024-09-20T17:37:06Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"fancyfredbot"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"If you game the benchmark then you always get found out by your users. Yet the practice remains common in hardware. Outright lies are uncommon but misleading and cherry picked numbers are pretty much standard practice.
The fact that misleading benchmarks don't even drive profit at Meta didn't seem to stop them doing the same thing, but perhaps this isn't very surprising. I imagine internal incentives are very similar.
Unlike the hardware companies though, gaming the benchmark in LLMs seems to involve making the actual performance worse, so perhaps there is more hope that the practice will fade away in this market."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Llama 4 Smells Bad"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://fastml.com/llama-4-smells-bad/"}},"_tags":["comment","author_fancyfredbot","story_43779915"],"author":"fancyfredbot","comment_text":"If you game the benchmark then you always get found out by your users. Yet the practice remains common in hardware. Outright lies are uncommon but misleading and cherry picked numbers are pretty much standard practice.
The fact that misleading benchmarks don't even drive profit at Meta didn't seem to stop them doing the same thing, but perhaps this isn't very surprising. I imagine internal incentives are very similar.
Unlike the hardware companies though, gaming the benchmark in LLMs seems to involve making the actual performance worse, so perhaps there is more hope that the practice will fade away in this market.","created_at":"2025-04-24T07:41:00Z","created_at_i":1745480460,"objectID":"43780162","parent_id":43779915,"story_id":43779915,"story_title":"Llama 4 Smells Bad","story_url":"https://fastml.com/llama-4-smells-bad/","updated_at":"2025-04-24T08:08:27Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nabakin"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"Exactly. Falcon-180b had a lot of hype at first but the community soon realized it was nearly worthless. Easily outperformed by smaller LLMs in the general case.
Now they are back and claiming their falcon-11b LLM outperforms Llama 3 8b. I already see a number of issues with this:
- falcon-11b is like 40% larger than Llama 3 8b so how can you compare them when they aren't in the same size class
- their claim seems to be based on automated benchmarks when it has long been clear that automated benchmarks are not enough to make that claim
- some of their automated benchmarks are wildly lower than Llama 3 8b's scores. It only beats Llama 3 8b on one benchmark and just barely. I can make an LLM does the best anyone has ever seen on one benchmark, but that doesn't mean my LLM is good. Far from it
- clickbait headline with knowingly premature claims because there has been zero human evaluation testing
- they claim their LLM is better than Llama 3 but completely ignore Llama 3 70b
Honestly, it annoys me how much attention tiiuae get when they haven't produced anything useful and continue this misleading clickbait."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Falcon 2"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.tii.ae/news/falcon-2-uaes-technology-innovation-institute-releases-new-ai-model-series-outperforming-metas"}},"_tags":["comment","author_nabakin","story_40344302"],"author":"nabakin","comment_text":"Exactly. Falcon-180b had a lot of hype at first but the community soon realized it was nearly worthless. Easily outperformed by smaller LLMs in the general case.
Now they are back and claiming their falcon-11b LLM outperforms Llama 3 8b. I already see a number of issues with this:
- falcon-11b is like 40% larger than Llama 3 8b so how can you compare them when they aren't in the same size class
- their claim seems to be based on automated benchmarks when it has long been clear that automated benchmarks are not enough to make that claim
- some of their automated benchmarks are wildly lower than Llama 3 8b's scores. It only beats Llama 3 8b on one benchmark and just barely. I can make an LLM does the best anyone has ever seen on one benchmark, but that doesn't mean my LLM is good. Far from it
- clickbait headline with knowingly premature claims because there has been zero human evaluation testing
- they claim their LLM is better than Llama 3 but completely ignore Llama 3 70b
Honestly, it annoys me how much attention tiiuae get when they haven't produced anything useful and continue this misleading clickbait.","created_at":"2024-05-13T18:13:15Z","created_at_i":1715623995,"objectID":"40346407","parent_id":40344589,"story_id":40344302,"story_title":"Falcon 2","story_url":"https://www.tii.ae/news/falcon-2-uaes-technology-innovation-institute-releases-new-ai-model-series-outperforming-metas","updated_at":"2024-09-20T17:03:07Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"PheonixPharts"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"The benchmarks are so easy to game it's not meaningful unless you see a major difference and even then it can be questionable. For example, did anyone read the Gemini paper[0]?
The team details the slight modification they made to the metric which is basically a tweak on maj@32 that worked in their favor. They choose to do the majority vote of 32 samples unless the probability of the highest sample was too low and then they chose greedy, they also had to learn what "too low" was. Sound convoluted and a bit contrived? They even show plots that clearly demonstrate they settled on the metric because it's the one where GPT-4 results remain they same, but Gemini wins.
Don't get me wrong, LLM evaluation is essential. But in order to do this you need to be very consistent between comparisons and even then the results can be misleading.
0. https://arxiv.org/pdf/2312.11805.pdf"},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["are"],"value":"Four new models that are benchmarking near or above GPT-4"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://simonwillison.net/2024/Mar/8/gpt-4-barrier/"}},"_tags":["comment","author_PheonixPharts","story_39643894"],"author":"PheonixPharts","children":[39645984],"comment_text":"The benchmarks are so easy to game it's not meaningful unless you see a major difference and even then it can be questionable. For example, did anyone read the Gemini paper[0]?
The team details the slight modification they made to the metric which is basically a tweak on maj@32 that worked in their favor. They choose to do the majority vote of 32 samples unless the probability of the highest sample was too low and then they chose greedy, they also had to learn what "too low" was. Sound convoluted and a bit contrived? They even show plots that clearly demonstrate they settled on the metric because it's the one where GPT-4 results remain they same, but Gemini wins.
Don't get me wrong, LLM evaluation is essential. But in order to do this you need to be very consistent between comparisons and even then the results can be misleading.
0. https://arxiv.org/pdf/2312.11805.pdf","created_at":"2024-03-08T18:37:25Z","created_at_i":1709923045,"objectID":"39644363","parent_id":39643894,"story_id":39643894,"story_title":"Four new models that are benchmarking near or above GPT-4","story_url":"https://simonwillison.net/2024/Mar/8/gpt-4-barrier/","updated_at":"2024-09-20T16:37:09Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"SirMaster"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"This is my biggest peeve when people say that LLMs are as capable as humans or that we have achieved AGI or are close or things like that.
But then when I get a subpar result, they always tell me I'm "prompting wrong". LLMs may be very capable of great human level output, but in my experience leave a LOT to be desired in terms of human level understanding of the question or prompt.
I think rating an LLM vs a human or AGI should include it's ability to understand a prompt like a human or like an averagely generally intelligent system should be able to.
Are there any benchmarks on that? Like how well LLMs do with misleading prompts or sparsely quantified prompts compared to one another?
Because if a good prompt is as important as people say, then the model's ability to understand a prompt or perhaps poor prompt could have a massive impact on its output."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"I want to wash my car. The car wash is 50 meters away. Should I walk or drive?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://mastodon.world/@knowmadd/116072773118828295"}},"_tags":["comment","author_SirMaster","story_47031580"],"author":"SirMaster","children":[47041646,47042401],"comment_text":"This is my biggest peeve when people say that LLMs are as capable as humans or that we have achieved AGI or are close or things like that.
But then when I get a subpar result, they always tell me I'm "prompting wrong". LLMs may be very capable of great human level output, but in my experience leave a LOT to be desired in terms of human level understanding of the question or prompt.
I think rating an LLM vs a human or AGI should include it's ability to understand a prompt like a human or like an averagely generally intelligent system should be able to.
Are there any benchmarks on that? Like how well LLMs do with misleading prompts or sparsely quantified prompts compared to one another?
Because if a good prompt is as important as people say, then the model's ability to understand a prompt or perhaps poor prompt could have a massive impact on its output.","created_at":"2026-02-16T17:59:46Z","created_at_i":1771264786,"objectID":"47038052","parent_id":47031882,"story_id":47031580,"story_title":"I want to wash my car. The car wash is 50 meters away. Should I walk or drive?","story_url":"https://mastodon.world/@knowmadd/116072773118828295","updated_at":"2026-03-05T23:33:54Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"imiric"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"Personal experience?
I don't care about benchmarks, nor LLMs' capability to solve puzzles. This is the first thing AI companies optimize their training for, which is misleading and practically false advertising.
I care about how good LLMs are for helping me with specific tasks. Do they generate code that on the surface appears correct, yet on closer inspection has security and performance issues, is unnecessarily complex, often doesn't even compile, which takes me more time to troubleshoot and fix than if I were to write it myself? Do they explain concepts confidently while being wrong, which I have no way of knowing unless I'm a domain expert? Do they repeat all these issues even after careful re-prompting and with all the contextual information they would need? Does all this waste my time more than it helps?
The answer is yes to all of the above.
So while we can argue whether LLMs can think and reason, my personal experience tells me that they absolutely cannot, and that any similarity to what humans can do is nothing but an illusion."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"\"The Illusion of Thinking\" \u2013 Thoughts on This Important Paper"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://hardcoresoftware.learningbyshipping.com/p/233-the-illusion-of-thinking-thoughts"}},"_tags":["comment","author_imiric","story_44234626"],"author":"imiric","comment_text":"Personal experience?
I don't care about benchmarks, nor LLMs' capability to solve puzzles. This is the first thing AI companies optimize their training for, which is misleading and practically false advertising.
I care about how good LLMs are for helping me with specific tasks. Do they generate code that on the surface appears correct, yet on closer inspection has security and performance issues, is unnecessarily complex, often doesn't even compile, which takes me more time to troubleshoot and fix than if I were to write it myself? Do they explain concepts confidently while being wrong, which I have no way of knowing unless I'm a domain expert? Do they repeat all these issues even after careful re-prompting and with all the contextual information they would need? Does all this waste my time more than it helps?
The answer is yes to all of the above.
So while we can argue whether LLMs can think and reason, my personal experience tells me that they absolutely cannot, and that any similarity to what humans can do is nothing but an illusion.","created_at":"2025-06-14T10:42:41Z","created_at_i":1749897761,"objectID":"44275535","parent_id":44275119,"story_id":44234626,"story_title":"\"The Illusion of Thinking\" \u2013 Thoughts on This Important Paper","story_url":"https://hardcoresoftware.learningbyshipping.com/p/233-the-illusion-of-thinking-thoughts","updated_at":"2025-06-14T14:26:56Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Swenrekcah"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"That is not true. Even ChatGPT understands how they are different, I won\u2019t paste the whole response but here are the differences it highlights:
Key differences:
1. Intent and harm:\n \u2022 VW\u2019s actions directly violated laws and had environmental and health consequences. Optimizing LLMs for chess benchmarks, while arguably misleading, doesn\u2019t have immediate real-world harms.\n 2. Scope: Chess-specific optimization is generally a transparent choice within AI research. It\u2019s not a hidden \u201cdefeat device\u201d but rather an explicit design goal.\n 3. Broader impact: LLMs fine-tuned for benchmarks often still retain general-purpose capabilities. They aren\u2019t necessarily \u201cbroken\u201d outside chess, whereas VW cars fundamentally failed to meet emissions standards."},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["llm"],"value":"Something weird is happening with LLMs and chess"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://dynomight.substack.com/p/chess"}},"_tags":["comment","author_Swenrekcah","story_42138289"],"author":"Swenrekcah","comment_text":"That is not true. Even ChatGPT understands how they are different, I won\u2019t paste the whole response but here are the differences it highlights:
Key differences:
1. Intent and harm:\n \u2022 VW\u2019s actions directly violated laws and had environmental and health consequences. Optimizing LLMs for chess benchmarks, while arguably misleading, doesn\u2019t have immediate real-world harms.\n 2. Scope: Chess-specific optimization is generally a transparent choice within AI research. It\u2019s not a hidden \u201cdefeat device\u201d but rather an explicit design goal.\n 3. Broader impact: LLMs fine-tuned for benchmarks often still retain general-purpose capabilities. They aren\u2019t necessarily \u201cbroken\u201d outside chess, whereas VW cars fundamentally failed to meet emissions standards.","created_at":"2024-11-15T14:10:12Z","created_at_i":1731679812,"objectID":"42147078","parent_id":42146749,"story_id":42138289,"story_title":"Something weird is happening with LLMs and chess","story_url":"https://dynomight.substack.com/p/chess","updated_at":"2024-11-15T20:11:08Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"cyrilou242"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"Sadly the paper uses for benchmarks datasets that:\n- are known to be pretty useless\n- contain mistakes \n- can be misleading with a naive F1-score measure. (to be fair they write "we looked at the F1-Score, under which both partial and full anomaly detection are considered correct identification" so this may be mitigated, but it's not clear)
See https://kdd-milets.github.io/milets2021/slides/Irrational%20...\nSo it's hard to take any benchmark from the paper seriously.\nThe paper is also ignoring any recent work (like > 2018) on univariate timeseries anomaly detection in the matrix profile space (eg MADRID).
The "practicality of usage" and conclusion sections are pretty correct though: it's expensive, slow, and no-shot is worthless if some other methods can train and infer in orders of magnitude less time.
It would have been interesting to see how the DETECTOR method performs when the LLM forecasting is replaced with some standard forecasting. (eg some auto ETS, if possible robust to anomalies in the training data). It looks like the natural follow up of this article is to remove the LLM altogether."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"MIT researchers use large language models to flag problems in complex systems"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://news.mit.edu/2024/researchers-use-large-language-models-to-flag-problems-0814"}},"_tags":["comment","author_cyrilou242","story_41253544"],"author":"cyrilou242","children":[41254686,41260565],"comment_text":"Sadly the paper uses for benchmarks datasets that:\n- are known to be pretty useless\n- contain mistakes \n- can be misleading with a naive F1-score measure. (to be fair they write "we looked at the F1-Score, under which both partial and full anomaly detection are considered correct identification" so this may be mitigated, but it's not clear)
See https://kdd-milets.github.io/milets2021/slides/Irrational%20...\nSo it's hard to take any benchmark from the paper seriously.\nThe paper is also ignoring any recent work (like > 2018) on univariate timeseries anomaly detection in the matrix profile space (eg MADRID).
The "practicality of usage" and conclusion sections are pretty correct though: it's expensive, slow, and no-shot is worthless if some other methods can train and infer in orders of magnitude less time.
It would have been interesting to see how the DETECTOR method performs when the LLM forecasting is replaced with some standard forecasting. (eg some auto ETS, if possible robust to anomalies in the training data). It looks like the natural follow up of this article is to remove the LLM altogether.","created_at":"2024-08-15T10:02:44Z","created_at_i":1723716164,"objectID":"41254432","parent_id":41253544,"story_id":41253544,"story_title":"MIT researchers use large language models to flag problems in complex systems","story_url":"https://news.mit.edu/2024/researchers-use-large-language-models-to-flag-problems-0814","updated_at":"2024-09-20T17:39:00Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"strofcon"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"I think you make a good point, benchmarks and metrics are indeed a better proxy for performance. Seems worth pointing out that, while "nowhere near half in [your] experience" are completely wrong, I don't take your word for it either. :-)
The trouble in my view is that the only way to know that the answers you're getting are accurate and not misleading is to study up on the answers elsewhere - which is a great habit to nurture, but is also precisely why these tools tend toward uselessness in their "general AI" bids. If I can't know how the answer was built, or how good that answer is, there's no point asking it - I'll just do my own reading and apply appropriate discernment as I go.
To be fair, hardly anyone does this today, nor did they before LLM-based chat bots... So it's a moot point, because society is largely doomed anyway. But a moot point can still be a valid one.
I also think the author makes a good point that we frequently confuse performance for competence. "It does a really good job at <X>!... or at least does a damn fine job of mimicking someone who acts like they do a really good job at <X>!"
By way of analogy, consider Elon Musk - by all appearances, he's a genius and is saving humanity - but by dint of his narcissism and largely smooth-brained approach to... well... everything... he's running all of us into an earlier planet-size grave than is necessary. His performance is fantastic, his competence is nonexistent."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Rodney Brooks on GPT-4"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://spectrum.ieee.org/gpt-4-calm-down"}},"_tags":["comment","author_strofcon","story_36017309"],"author":"strofcon","children":[36017756],"comment_text":"I think you make a good point, benchmarks and metrics are indeed a better proxy for performance. Seems worth pointing out that, while "nowhere near half in [your] experience" are completely wrong, I don't take your word for it either. :-)
The trouble in my view is that the only way to know that the answers you're getting are accurate and not misleading is to study up on the answers elsewhere - which is a great habit to nurture, but is also precisely why these tools tend toward uselessness in their "general AI" bids. If I can't know how the answer was built, or how good that answer is, there's no point asking it - I'll just do my own reading and apply appropriate discernment as I go.
To be fair, hardly anyone does this today, nor did they before LLM-based chat bots... So it's a moot point, because society is largely doomed anyway. But a moot point can still be a valid one.
I also think the author makes a good point that we frequently confuse performance for competence. "It does a really good job at <X>!... or at least does a damn fine job of mimicking someone who acts like they do a really good job at <X>!"
By way of analogy, consider Elon Musk - by all appearances, he's a genius and is saving humanity - but by dint of his narcissism and largely smooth-brained approach to... well... everything... he's running all of us into an earlier planet-size grave than is necessary. His performance is fantastic, his competence is nonexistent.","created_at":"2023-05-21T02:52:56Z","created_at_i":1684637576,"objectID":"36017609","parent_id":36017443,"story_id":36017309,"story_title":"Rodney Brooks on GPT-4","story_url":"https://spectrum.ieee.org/gpt-4-calm-down","updated_at":"2024-09-20T14:08:26Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ftxbro"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"the most misleading things i saw in llm ai were the reports of performances of local models. several independent reports of various models were saying better than gpt-3.5 or even gpt-4 and it was never true. maybe they are doing their benchmarks in a silly way. it made me not trust these analyses of local models.
EDIT: everyone in that reddit calling them out for this. i didn't realize it was such a meme lol"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"WizardLM-30B v1.0: beats Guanaco-65B"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://old.reddit.com/r/LocalLLaMA/comments/142iw20/official_wizardlm30b_v10_released_can_beat/"}},"_tags":["comment","author_ftxbro","story_36216650"],"author":"ftxbro","comment_text":"the most misleading things i saw in llm ai were the reports of performances of local models. several independent reports of various models were saying better than gpt-3.5 or even gpt-4 and it was never true. maybe they are doing their benchmarks in a silly way. it made me not trust these analyses of local models.
EDIT: everyone in that reddit calling them out for this. i didn't realize it was such a meme lol","created_at":"2023-06-06T18:10:01Z","created_at_i":1686075001,"objectID":"36216830","parent_id":36216650,"story_id":36216650,"story_title":"WizardLM-30B v1.0: beats Guanaco-65B","story_url":"https://old.reddit.com/r/LocalLLaMA/comments/142iw20/official_wizardlm30b_v10_released_can_beat/","updated_at":"2024-09-20T14:11:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"famouswaffles"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":">If the benchmark is not representative of normal usage* then the benchmark and the plot being shown are not useful at all from a user/business perspective and the focus on the breakthrough scores of o3-low and o3-high in ARC-AGI would be highly misleading.
ARC is a very hyped benchmark in the industry so letting us know the results is something any company would do whether it had a direct representation on normal usage or not.
>Even if it is pure marketing they expect people to draw conclusions using the perf/cost plot from Arc.
Again, people care about ARC, they don't care doing the things ARC questions ask. That it is un-economical to pay the price to use o3 for ARC does not mean it would be un-economical to do so for the tasks people actually want to use LLMs for. What does 3x the performance in say coding mean? You really think companies/users wouldn't put up with the increased price for that? You think they have Mturkers to turn to like they do with ARC?
ARC is literally the quintessential 'easy for humans, hard for ai' benchmark. Even if you discard the 'difficulty to price won't scale the same' argument, it makes no sense to use it for an economics comparison."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"GPT-5 is behind schedule"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.wsj.com/tech/ai/openai-gpt5-orion-delays-639e7693"}},"_tags":["comment","author_famouswaffles","story_42485938"],"author":"famouswaffles","children":[42492129],"comment_text":">If the benchmark is not representative of normal usage* then the benchmark and the plot being shown are not useful at all from a user/business perspective and the focus on the breakthrough scores of o3-low and o3-high in ARC-AGI would be highly misleading.
ARC is a very hyped benchmark in the industry so letting us know the results is something any company would do whether it had a direct representation on normal usage or not.
>Even if it is pure marketing they expect people to draw conclusions using the perf/cost plot from Arc.
Again, people care about ARC, they don't care doing the things ARC questions ask. That it is un-economical to pay the price to use o3 for ARC does not mean it would be un-economical to do so for the tasks people actually want to use LLMs for. What does 3x the performance in say coding mean? You really think companies/users wouldn't put up with the increased price for that? You think they have Mturkers to turn to like they do with ARC?
ARC is literally the quintessential 'easy for humans, hard for ai' benchmark. Even if you discard the 'difficulty to price won't scale the same' argument, it makes no sense to use it for an economics comparison.","created_at":"2024-12-23T03:42:33Z","created_at_i":1734925353,"objectID":"42491537","parent_id":42491089,"story_id":42485938,"story_title":"GPT-5 is behind schedule","story_url":"https://www.wsj.com/tech/ai/openai-gpt5-orion-delays-639e7693","updated_at":"2025-11-10T20:40:58Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"LoadingALIAS"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"Karolin, I\u2019ll answer this directly, and try to stay on track, because several claims here are materially wrong.
First things first, I am not claiming I've had any third-party audit; I'm not claiming I've been FIPS validated, or telling anyone \u201cuse this blindly in high-assurance production.\u201d If that is your bar, then yes: do not use rscrypto yet. That is a fair warning... and it's plastered everywhere. I almost feel like you didn't read the codebase, the thread here, etc.
Now, to your constant-time criticism - it's just not correct. It's wrong. It's misleading. It shouldn't come from someone leading Rosenpass.
\u201cNot a global constant-time claim\u201d does not mean \u201csecret-bearing cryptographic ops branch on secrets.\u201d I've said this twice now - verbatim. It means I am refusing to make a fake blanket claim over APIs where constant-time behavior is either irrelevant, impossible, or the wrong security property.
DER parsing is not supposed to be constant-time over DER structure. PHC string decoding is not supposed to be constant-time over ASCII syntax. OS randomness is not a constant-time primitive. RSA public verification/encryption operates on public inputs. Rejection of public length/shape before entering a primitive boundary is NOT a secret-dependent timing leak. Argon2d intentionally has data-dependent memory access; treating it as a blanket CT primitive would be wrong - it's a lie at that point.
That list was a boundary, Karolin, not an admission of insecure primitives.
The MAC/hash point is also mixing categories. A MAC verification claim is about the keyed construction and verification surface: no secret-dependent behavior in the keyed path that matters for the construction, opaque failure, and constant-time tag comparison. An unkeyed hash API as a whole does not get the same global claim because message length, streaming shape, finalization shape, feature-dispatched kernels, and non-crypto hash/checksum APIs are public-input machinery. \u201cHMAC uses a hash\u201d does not imply \u201cthe entire hash API must be globally constant-time under every possible use.\u201d
On \u201calgo/profile negotiation\u201d: this is not TLS-style negotiation. It is closed protocol identifier mapping for RSA profiles: JWT/COSE/TLS/X.509 identifiers are parsed into EXPLICIT supported profile enums, and unsupported/confused algos are rejected. If the word \u201cnegotiation\u201d suggested downgrade-prone protocol behavior, I should fix the language, sure, but the feature is not an open negotiation mechanism.
With respect to the perf, if you believe a benchmark win comes from a specific missing constant-time property, name the primitive, input class, backend, and compared impl. \u201cIt might be faster because it is insecure\u201d is not evidence... it's negativity and hyperbole.
Also, every single line of this codebase is my responsibility. My usage of an LLM is not a substitute for review, tests, vectors, fuzzing, Miri, and/or a third-party audit. If you're insinuating that using an LLM today is a vulnerability report - I'm afraid you're falling behind the best engineers in the world and are sorely mistaken.
Finally, and this is the one that really gets under my skin, which is likely your goal given your very public history of such... my putting code in public doesn't force cryptographers to review it. It invites scrutiny. If there is duplication that materially increases audit cost, point to it and I will remove or justify it. If there is a side channel, GIVE ME THE PATH and I will treat it as a security issue. But \u201cdo not publish until the community has reviewed it\u201d is backwards: review cannot happen against code that is not visible. You're plastered all across the Internet championing this exact maxim, are you not? Did I not read an interview this morning where you went on about the importance of OSS crypto?
Please, take some time to review the codebase or don't; that's up to you... but don't come in here and trash my work improving and unifying well known primitives. Also, this isn't a whimsical codebase vibe coded by Claude overnight. This is a year of my life reading, understanding, and improving the inefficiencies in primitives I've used for years.
The entire point of a pre-v1 review by the community is the same exact thing OSS engineers/contributors have done for decades. I'm looking for the community to point out glaring issues, errors, API shapes, code smells, etc. You came in here with a bunch of incorrect claims and emotion. This codebase isn't perfect, but I assure you - it will be, and it will lead Rust by default becasue there simple will not be a better option.
Happy to talk one on one, or we could put something together where others have a chance to chime in. Either way, have a good one."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Rscrypto, pure-Rust crypto with industry leading public benches"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/loadingalias/rscrypto"}},"_tags":["comment","author_LoadingALIAS","story_48386343"],"author":"LoadingALIAS","comment_text":"Karolin, I\u2019ll answer this directly, and try to stay on track, because several claims here are materially wrong.
First things first, I am not claiming I've had any third-party audit; I'm not claiming I've been FIPS validated, or telling anyone \u201cuse this blindly in high-assurance production.\u201d If that is your bar, then yes: do not use rscrypto yet. That is a fair warning... and it's plastered everywhere. I almost feel like you didn't read the codebase, the thread here, etc.
Now, to your constant-time criticism - it's just not correct. It's wrong. It's misleading. It shouldn't come from someone leading Rosenpass.
\u201cNot a global constant-time claim\u201d does not mean \u201csecret-bearing cryptographic ops branch on secrets.\u201d I've said this twice now - verbatim. It means I am refusing to make a fake blanket claim over APIs where constant-time behavior is either irrelevant, impossible, or the wrong security property.
DER parsing is not supposed to be constant-time over DER structure. PHC string decoding is not supposed to be constant-time over ASCII syntax. OS randomness is not a constant-time primitive. RSA public verification/encryption operates on public inputs. Rejection of public length/shape before entering a primitive boundary is NOT a secret-dependent timing leak. Argon2d intentionally has data-dependent memory access; treating it as a blanket CT primitive would be wrong - it's a lie at that point.
That list was a boundary, Karolin, not an admission of insecure primitives.
The MAC/hash point is also mixing categories. A MAC verification claim is about the keyed construction and verification surface: no secret-dependent behavior in the keyed path that matters for the construction, opaque failure, and constant-time tag comparison. An unkeyed hash API as a whole does not get the same global claim because message length, streaming shape, finalization shape, feature-dispatched kernels, and non-crypto hash/checksum APIs are public-input machinery. \u201cHMAC uses a hash\u201d does not imply \u201cthe entire hash API must be globally constant-time under every possible use.\u201d
On \u201calgo/profile negotiation\u201d: this is not TLS-style negotiation. It is closed protocol identifier mapping for RSA profiles: JWT/COSE/TLS/X.509 identifiers are parsed into EXPLICIT supported profile enums, and unsupported/confused algos are rejected. If the word \u201cnegotiation\u201d suggested downgrade-prone protocol behavior, I should fix the language, sure, but the feature is not an open negotiation mechanism.
With respect to the perf, if you believe a benchmark win comes from a specific missing constant-time property, name the primitive, input class, backend, and compared impl. \u201cIt might be faster because it is insecure\u201d is not evidence... it's negativity and hyperbole.
Also, every single line of this codebase is my responsibility. My usage of an LLM is not a substitute for review, tests, vectors, fuzzing, Miri, and/or a third-party audit. If you're insinuating that using an LLM today is a vulnerability report - I'm afraid you're falling behind the best engineers in the world and are sorely mistaken.
Finally, and this is the one that really gets under my skin, which is likely your goal given your very public history of such... my putting code in public doesn't force cryptographers to review it. It invites scrutiny. If there is duplication that materially increases audit cost, point to it and I will remove or justify it. If there is a side channel, GIVE ME THE PATH and I will treat it as a security issue. But \u201cdo not publish until the community has reviewed it\u201d is backwards: review cannot happen against code that is not visible. You're plastered all across the Internet championing this exact maxim, are you not? Did I not read an interview this morning where you went on about the importance of OSS crypto?
Please, take some time to review the codebase or don't; that's up to you... but don't come in here and trash my work improving and unifying well known primitives. Also, this isn't a whimsical codebase vibe coded by Claude overnight. This is a year of my life reading, understanding, and improving the inefficiencies in primitives I've used for years.
The entire point of a pre-v1 review by the community is the same exact thing OSS engineers/contributors have done for decades. I'm looking for the community to point out glaring issues, errors, API shapes, code smells, etc. You came in here with a bunch of incorrect claims and emotion. This codebase isn't perfect, but I assure you - it will be, and it will lead Rust by default becasue there simple will not be a better option.
Happy to talk one on one, or we could put something together where others have a chance to chime in. Either way, have a good one.","created_at":"2026-06-04T14:51:20Z","created_at_i":1780584680,"objectID":"48399569","parent_id":48394475,"story_id":48386343,"story_title":"Show HN: Rscrypto, pure-Rust crypto with industry leading public benches","story_url":"https://github.com/loadingalias/rscrypto","updated_at":"2026-06-05T14:16:00Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"famouswaffles"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":">I'm not expecting novel scientific theories today. What I am expecting are signs and hints of such genius.
Like I said, what exactly would you be expecting to see with the capabilities that exist today ? It's not a gotcha, it's a genuine question.
>And, I'm sorry, I don't buy the excuse that these tools are not "yet" as capable as the best and brightest humans.
There's nothing to buy or not buy. They simply aren't. They are unable to do a lot of the things these people do. You can't slot an LLM in place of most knowledge workers and expect everything to be fine and dandy. There's no ambiguity on that.
>They contain the sum of human knowledge, far more than any individual human in history.
It's not really the total sum of human knowledge but let's set that aside. Yeah so ? Einstein, Newton, Von Newman. None of these guys were privy to some super secret knowledge their contemporaries weren't so it's obviously not simply a matter of more knowledge.
>Are they not intelligent, capable of thinking and reasoning?
Yeah they are. And so are humans. So were the peers of all those guys. So why are only a few able to see the next step ? It's not just about knowledge, and intelligence lives in degrees/is a gradient.
>If all this is true, surely we should be seeing incredible results produced by this technology. If not by itself, then surely by "amplifying" the work of the best and brightest humans.
Yeah and that exists. Terence Tao has shared a lot of his (and his peers) experiences on the matter.
https://mathstodon.xyz/@tao/115306424727150237
https://mathstodon.xyz/@tao/115420236285085121
https://mathstodon.xyz/@tao/115416208975810074
>And yet... All we have to show for it are some very good applications of pattern matching and statistics, a bunch of gamed and misleading benchmarks and leaderboards, a whole lot of tech demos, solutions in search of a problem, and the very real problem of flooding us with even more spam, scams, disinformation, and devaluing human work with low-effort garbage.
Well it's a good thing that's not true then"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"A new Google model is nearly perfect on automated handwriting recognition"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://generativehistory.substack.com/p/has-google-quietly-solved-two-of"}},"_tags":["comment","author_famouswaffles","story_45887262"],"author":"famouswaffles","children":[45936418],"comment_text":">I'm not expecting novel scientific theories today. What I am expecting are signs and hints of such genius.
Like I said, what exactly would you be expecting to see with the capabilities that exist today ? It's not a gotcha, it's a genuine question.
>And, I'm sorry, I don't buy the excuse that these tools are not "yet" as capable as the best and brightest humans.
There's nothing to buy or not buy. They simply aren't. They are unable to do a lot of the things these people do. You can't slot an LLM in place of most knowledge workers and expect everything to be fine and dandy. There's no ambiguity on that.
>They contain the sum of human knowledge, far more than any individual human in history.
It's not really the total sum of human knowledge but let's set that aside. Yeah so ? Einstein, Newton, Von Newman. None of these guys were privy to some super secret knowledge their contemporaries weren't so it's obviously not simply a matter of more knowledge.
>Are they not intelligent, capable of thinking and reasoning?
Yeah they are. And so are humans. So were the peers of all those guys. So why are only a few able to see the next step ? It's not just about knowledge, and intelligence lives in degrees/is a gradient.
>If all this is true, surely we should be seeing incredible results produced by this technology. If not by itself, then surely by "amplifying" the work of the best and brightest humans.
Yeah and that exists. Terence Tao has shared a lot of his (and his peers) experiences on the matter.
https://mathstodon.xyz/@tao/115306424727150237
https://mathstodon.xyz/@tao/115420236285085121
https://mathstodon.xyz/@tao/115416208975810074
>And yet... All we have to show for it are some very good applications of pattern matching and statistics, a bunch of gamed and misleading benchmarks and leaderboards, a whole lot of tech demos, solutions in search of a problem, and the very real problem of flooding us with even more spam, scams, disinformation, and devaluing human work with low-effort garbage.
Well it's a good thing that's not true then","created_at":"2025-11-15T03:49:28Z","created_at_i":1763178568,"objectID":"45934910","parent_id":45934710,"story_id":45887262,"story_title":"A new Google model is nearly perfect on automated handwriting recognition","story_url":"https://generativehistory.substack.com/p/has-google-quietly-solved-two-of","updated_at":"2026-03-05T22:59:48Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"imiric"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"> LLMs are great, but they're not (yet?) as capable as our best and brightest (and in many ways, lag behind the average human) in most respects, so why would you expect such genius now ?
I'm not expecting novel scientific theories today. What I am expecting are signs and hints of such genius. Something that points in the direction that all tech CEOs are claiming we're headed in. So far I haven't seen any of this yet.
And, I'm sorry, I don't buy the excuse that these tools are not "yet" as capable as the best and brightest humans. They contain the sum of human knowledge, far more than any individual human in history. Are they not intelligent, capable of thinking and reasoning? Are we not at the verge of superintelligence[1]?
> we have recently built systems that are smarter than people in many ways, and are able to significantly amplify the output of people using them.
If all this is true, surely we should be seeing incredible results produced by this technology. If not by itself, then surely by "amplifying" the work of the best and brightest humans.
And yet... All we have to show for it are some very good applications of pattern matching and statistics, a bunch of gamed and misleading benchmarks and leaderboards, a whole lot of tech demos, solutions in search of a problem, and the very real problem of flooding us with even more spam, scams, disinformation, and devaluing human work with low-effort garbage.
[1]: https://blog.samaltman.com/the-gentle-singularity"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"A new Google model is nearly perfect on automated handwriting recognition"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://generativehistory.substack.com/p/has-google-quietly-solved-two-of"}},"_tags":["comment","author_imiric","story_45887262"],"author":"imiric","children":[45934910],"comment_text":"> LLMs are great, but they're not (yet?) as capable as our best and brightest (and in many ways, lag behind the average human) in most respects, so why would you expect such genius now ?
I'm not expecting novel scientific theories today. What I am expecting are signs and hints of such genius. Something that points in the direction that all tech CEOs are claiming we're headed in. So far I haven't seen any of this yet.
And, I'm sorry, I don't buy the excuse that these tools are not "yet" as capable as the best and brightest humans. They contain the sum of human knowledge, far more than any individual human in history. Are they not intelligent, capable of thinking and reasoning? Are we not at the verge of superintelligence[1]?
> we have recently built systems that are smarter than people in many ways, and are able to significantly amplify the output of people using them.
If all this is true, surely we should be seeing incredible results produced by this technology. If not by itself, then surely by "amplifying" the work of the best and brightest humans.
And yet... All we have to show for it are some very good applications of pattern matching and statistics, a bunch of gamed and misleading benchmarks and leaderboards, a whole lot of tech demos, solutions in search of a problem, and the very real problem of flooding us with even more spam, scams, disinformation, and devaluing human work with low-effort garbage.
[1]: https://blog.samaltman.com/the-gentle-singularity","created_at":"2025-11-15T03:05:46Z","created_at_i":1763175946,"objectID":"45934710","parent_id":45934067,"story_id":45887262,"story_title":"A new Google model is nearly perfect on automated handwriting recognition","story_url":"https://generativehistory.substack.com/p/has-google-quietly-solved-two-of","updated_at":"2026-03-05T22:59:48Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"julianhuang"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"1. The "master geoguesser" is a bit misleading--as mentioned in his blog post, there are players far better than him, and he is certainly not the bar for human supremacy. Probably analogous to a 1400-1800 elo chess player.\n2. o3 isn't the best model at playing GeoGuessr, Gemini 1.5 & 2.5 solidly beat it out--for those interested, check out my friend's benchmark (https://geobench.org/) and blog post (https://ccmdi.com/blog/GeoBench) detailing interesting model explanations.\n3. In the post, he only tests on one game--o3's average score over 100 locations (20 5-location games) was 19,290, far lower than the 23,179 in the game.\nModel geolocation capabilities are really important to keep track of, but the specific blog post in question isn't anything out of the ordinary. LLMs are making geolocation abilities much more accessible, but still fall short compared to 1. top GeoGuessr players playing GeoGuessr (only google streetview coverage, without web search) and 2. professional geolocators, who are proficient at using a wide variety of software/search. I.e., if the CIA wanted to find someone using an image, LLMs would not provide them any unique ability to do so as opposed to someone like Rainbolt"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"O3 beats a master-level GeoGuessr player, even with fake EXIF data"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://sampatt.com/blog/2025-04-28-can-o3-beat-a-geoguessr-master"}},"_tags":["comment","author_julianhuang","story_43835044"],"author":"julianhuang","children":[43835360,43835368,43836362],"comment_text":"1. The "master geoguesser" is a bit misleading--as mentioned in his blog post, there are players far better than him, and he is certainly not the bar for human supremacy. Probably analogous to a 1400-1800 elo chess player.\n2. o3 isn't the best model at playing GeoGuessr, Gemini 1.5 & 2.5 solidly beat it out--for those interested, check out my friend's benchmark (https://geobench.org/) and blog post (https://ccmdi.com/blog/GeoBench) detailing interesting model explanations.\n3. In the post, he only tests on one game--o3's average score over 100 locations (20 5-location games) was 19,290, far lower than the 23,179 in the game.\nModel geolocation capabilities are really important to keep track of, but the specific blog post in question isn't anything out of the ordinary. LLMs are making geolocation abilities much more accessible, but still fall short compared to 1. top GeoGuessr players playing GeoGuessr (only google streetview coverage, without web search) and 2. professional geolocators, who are proficient at using a wide variety of software/search. I.e., if the CIA wanted to find someone using an image, LLMs would not provide them any unique ability to do so as opposed to someone like Rainbolt","created_at":"2025-04-29T17:06:21Z","created_at_i":1745946381,"objectID":"43835279","parent_id":43835044,"story_id":43835044,"story_title":"O3 beats a master-level GeoGuessr player, even with fake EXIF data","story_url":"https://sampatt.com/blog/2025-04-28-can-o3-beat-a-geoguessr-master","updated_at":"2025-04-29T21:32:12Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Topfi"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"Here\u2019s their response: https://xcancel.com/cognition_labs/status/186735152103553069...
This situation is both embarrassing and difficult to excuse given the severity. It\u2019s especially puzzling since they\u2019ve been testing this for about a year. Something of this nature should have been caught and feels amateurish. That said, I\u2019ll give them credit for reacting quickly to address the issue and can understand that, given startup pressures, mistakes can happen and bigger organizations with more experience have made similar mistake. Not to justify, but I feel providing context is fair.
But to provide more context, let us remember their months of dubious claims about their "models" supposed superiority. They\u2019ve repeatedly asserted that Devin outperforms all other LLM implementation for coding tasks purely in quality and ability, actively identifies bugs, and independently fixes them. In the public access announcement thread [0], several early access users highlighted this as a key difference between Devin and tools like Cursor/VSCode plugins for LLMs. They claimed Devin could be "told" to independently "search for issues" and then create pull requests, effectively functioning like a "virtual Junior Dev" assigned to tasks.
Clearly, that\u2019s not the case if they themselves don't seem to rely on Devin?
The crux of the problem lies beyond their past credibility issues or the tragic "Leetcode genius dropout" theatrics (something that should have gone out of style long before SBF started playing League of Legends during investor calls). My primary criticism of Devin, purely as a product, is the misleading presentation. They try so hard to position themselves as superior in raw coding performance, not just implementation of other models with their toolset, yet the code examples and pull requests don\u2019t support that claim at all.
If, as I suspect, Devin is just a set of tools to give OpenAIs API more access to ones codebase, that is totally fine, less then promised, but still something useful and can add significant value, as seen with tools like Cursor. While, whether LLMs at this stage should operate with the level of autonomy Devin seems to target is debatable [1], I can see the concept behind Devin having a future. If Devins team simply stated their aim was to create a solid foundation for LLMs to interact with code beyond traditional IDEs, priced it appropriately and avoided any notion that they aren't reselling existing models, I feel they'd do a lot better in the long run and I'd be a customer.
There is no shame in relying on another companies API something said company doesn't, as seen with Curor. Devin should position themselves in the same manner as Cursor. Acting like the model part of the equation is what makes Devin superior, that is what opens them to criticism.
Well, that and their previous fake demos, lack of benchmarks or whitepapers, charging more than all competitors combined, having rather embarrassing security issues and how they approached the media up to this point.
Alternatively, maybe I am wrong and those previous assertions of being superior purely in coding performance are credible. In that case, they could silence any doubts by proving their system isn\u2019t merely an API wrapper by releasing transparent performance metrics [2].
[0] https://news.ycombinator.com/item?id=42378994
[1] I feel LLMs in coding at this stage are roughly at the same point driver assistance in modern vehicles tends to be, just good enough that users could be lead into a false sense of security, stop paying attention and end up with issues. A solution like Devin or some "FSD" solution could be reliable for multiple hours, then suddenly produce faulty code or disengage, making it harder to react straight away. In both cases, I feel that UX at this stage should lean into ensuring consistent verification, something that the current crop of VSCode plugins or Cursor do a decent job of by consistently highlighting diffs.
[2] Example how this is done: https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bb..."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Streamer discovers major vulnerability in Cognition's Devin live on air"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://twitter.com/TheMidasProj/status/1867318553046921376"}},"_tags":["comment","author_Topfi","story_42404132"],"author":"Topfi","comment_text":"Here\u2019s their response: https://xcancel.com/cognition_labs/status/186735152103553069...
This situation is both embarrassing and difficult to excuse given the severity. It\u2019s especially puzzling since they\u2019ve been testing this for about a year. Something of this nature should have been caught and feels amateurish. That said, I\u2019ll give them credit for reacting quickly to address the issue and can understand that, given startup pressures, mistakes can happen and bigger organizations with more experience have made similar mistake. Not to justify, but I feel providing context is fair.
But to provide more context, let us remember their months of dubious claims about their "models" supposed superiority. They\u2019ve repeatedly asserted that Devin outperforms all other LLM implementation for coding tasks purely in quality and ability, actively identifies bugs, and independently fixes them. In the public access announcement thread [0], several early access users highlighted this as a key difference between Devin and tools like Cursor/VSCode plugins for LLMs. They claimed Devin could be "told" to independently "search for issues" and then create pull requests, effectively functioning like a "virtual Junior Dev" assigned to tasks.
Clearly, that\u2019s not the case if they themselves don't seem to rely on Devin?
The crux of the problem lies beyond their past credibility issues or the tragic "Leetcode genius dropout" theatrics (something that should have gone out of style long before SBF started playing League of Legends during investor calls). My primary criticism of Devin, purely as a product, is the misleading presentation. They try so hard to position themselves as superior in raw coding performance, not just implementation of other models with their toolset, yet the code examples and pull requests don\u2019t support that claim at all.
If, as I suspect, Devin is just a set of tools to give OpenAIs API more access to ones codebase, that is totally fine, less then promised, but still something useful and can add significant value, as seen with tools like Cursor. While, whether LLMs at this stage should operate with the level of autonomy Devin seems to target is debatable [1], I can see the concept behind Devin having a future. If Devins team simply stated their aim was to create a solid foundation for LLMs to interact with code beyond traditional IDEs, priced it appropriately and avoided any notion that they aren't reselling existing models, I feel they'd do a lot better in the long run and I'd be a customer.
There is no shame in relying on another companies API something said company doesn't, as seen with Curor. Devin should position themselves in the same manner as Cursor. Acting like the model part of the equation is what makes Devin superior, that is what opens them to criticism.
Well, that and their previous fake demos, lack of benchmarks or whitepapers, charging more than all competitors combined, having rather embarrassing security issues and how they approached the media up to this point.
Alternatively, maybe I am wrong and those previous assertions of being superior purely in coding performance are credible. In that case, they could silence any doubts by proving their system isn\u2019t merely an API wrapper by releasing transparent performance metrics [2].
[0] https://news.ycombinator.com/item?id=42378994
[1] I feel LLMs in coding at this stage are roughly at the same point driver assistance in modern vehicles tends to be, just good enough that users could be lead into a false sense of security, stop paying attention and end up with issues. A solution like Devin or some "FSD" solution could be reliable for multiple hours, then suddenly produce faulty code or disengage, making it harder to react straight away. In both cases, I feel that UX at this stage should lean into ensuring consistent verification, something that the current crop of VSCode plugins or Cursor do a decent job of by consistently highlighting diffs.
[2] Example how this is done: https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bb...","created_at":"2024-12-13T00:12:24Z","created_at_i":1734048744,"objectID":"42404760","parent_id":42404132,"story_id":42404132,"story_title":"Streamer discovers major vulnerability in Cognition's Devin live on air","story_url":"https://twitter.com/TheMidasProj/status/1867318553046921376","updated_at":"2024-12-15T01:27:10Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"JoshMandel"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["llm","benchmarks","are","misleading"],"value":"There's so much good stuff here, and I agree it's an important message for you to get across.
I think trying to convey these ideas through a quantitative benchmark result (particularly a benchmark which has a clear common interpretation that you're essentially redefining) risks 1) misleading readers, and 2) failing to convey the rich and detailed analysis you've included here in your HN comment.
I'd suggest you restrict your quantitative PubMedQA analysis to report previously published numbers for other models (so you're not in the role of having to defend choices that might cripple other models) or a very straightforward log probs analysis if no outside numbers are available (making it clear which numbers you've produced vs sourced externally). Then separately explain that many of the small models with high benchmark scores exhibit poor instruction following capabilities (which will not be a surprise for many readers, since these models aren't necessary tuned or evaluated for that), and you can make the point that some of them are so poor at instruction following that they're very hard to deploy in contexts that require instruction following; you could even demonstrate that they're only able to follow an instructions to "conclude answers with 'Final Answer: [ABCDE]'" on x% of questions, given a standard prompt that you've created and published. In other words, if it's clear that the problem is in instruction following, analyze that.
(Not all abstraction pipelines leveraging an LLM need it to exhibit instruction following, and in your own case, I'm not sure you can claim that your model follows instructions well on the basis of its PubMedQA or abstraction performance, since you've fine tuned on prompt,answer pairs in both domains. You'd need a different baseline for comparison to really explore this claim.)
Then I'd suggest creating a detailed table of wrong/surprising stuff that frontier models don't understand about healthcare data, but which your model does understand. Categorize them, show examples in the table, and explain them in narrative much like you've done here."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"LLMD: A Large Language Model for Interpreting Longitudinal Medical Records"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://arxiv.org/abs/2410.12860"}},"_tags":["comment","author_JoshMandel","story_41878959"],"author":"JoshMandel","comment_text":"There's so much good stuff here, and I agree it's an important message for you to get across.
I think trying to convey these ideas through a quantitative benchmark result (particularly a benchmark which has a clear common interpretation that you're essentially redefining) risks 1) misleading readers, and 2) failing to convey the rich and detailed analysis you've included here in your HN comment.
I'd suggest you restrict your quantitative PubMedQA analysis to report previously published numbers for other models (so you're not in the role of having to defend choices that might cripple other models) or a very straightforward log probs analysis if no outside numbers are available (making it clear which numbers you've produced vs sourced externally). Then separately explain that many of the small models with high benchmark scores exhibit poor instruction following capabilities (which will not be a surprise for many readers, since these models aren't necessary tuned or evaluated for that), and you can make the point that some of them are so poor at instruction following that they're very hard to deploy in contexts that require instruction following; you could even demonstrate that they're only able to follow an instructions to "conclude answers with 'Final Answer: [ABCDE]'" on x% of questions, given a standard prompt that you've created and published. In other words, if it's clear that the problem is in instruction following, analyze that.
(Not all abstraction pipelines leveraging an LLM need it to exhibit instruction following, and in your own case, I'm not sure you can claim that your model follows instructions well on the basis of its PubMedQA or abstraction performance, since you've fine tuned on prompt,answer pairs in both domains. You'd need a different baseline for comparison to really explore this claim.)
Then I'd suggest creating a detailed table of wrong/surprising stuff that frontier models don't understand about healthcare data, but which your model does understand. Categorize them, show examples in the table, and explain them in narrative much like you've done here.","created_at":"2024-10-20T13:22:22Z","created_at_i":1729430542,"objectID":"41895211","parent_id":41890098,"story_id":41878959,"story_title":"LLMD: A Large Language Model for Interpreting Longitudinal Medical Records","story_url":"https://arxiv.org/abs/2410.12860","updated_at":"2024-10-20T13:26:07Z"},{"_highlightResult":{"author":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["llm"],"value":"llm_nerd"},"comment_text":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["benchmarks","are","misleading"],"value":">What else was the "Out-of-the-box" Windows installation trying to do while they were running the benchmark?
It's trying to maximize the performance to power usage ratio out of the box. The default power profile is very conservative, which on a laptop means better battery life and reduced heat. Given that this comparison didn't do even rudimentary power consumption, heat, or CPU frequency scaling measures, it is overwhelmingly likely as the difference.
> there are dozens of things clamoring for the attention of the scheduler and using CPU
Linux machines have hundreds of processes too. It simply doesn't matter in 2024. That isn't a factor and isn't relevant, nor is the scheduler to blame.
This is a silly misleading benchmark that is de facto clickbait. Pretty surprisedto see it doing well here."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Windows 11 vs. Ubuntu 24.10 Performance For Intel Core Ultra 7 Lunar Lake"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.phoronix.com/review/lunar-lake-windows-linux/5"}},"_tags":["comment","author_llm_nerd","story_41788557"],"author":"llm_nerd","comment_text":">What else was the "Out-of-the-box" Windows installation trying to do while they were running the benchmark?
It's trying to maximize the performance to power usage ratio out of the box. The default power profile is very conservative, which on a laptop means better battery life and reduced heat. Given that this comparison didn't do even rudimentary power consumption, heat, or CPU frequency scaling measures, it is overwhelmingly likely as the difference.
> there are dozens of things clamoring for the attention of the scheduler and using CPU
Linux machines have hundreds of processes too. It simply doesn't matter in 2024. That isn't a factor and isn't relevant, nor is the scheduler to blame.
This is a silly misleading benchmark that is de facto clickbait. Pretty surprisedto see it doing well here.","created_at":"2024-10-09T15:46:59Z","created_at_i":1728488819,"objectID":"41789196","parent_id":41788876,"story_id":41788557,"story_title":"Windows 11 vs. Ubuntu 24.10 Performance For Intel Core Ultra 7 Lunar Lake","story_url":"https://www.phoronix.com/review/lunar-lake-windows-linux/5","updated_at":"2024-10-09T17:27:42Z"}],"hitsPerPage":20,"nbHits":22,"nbPages":2,"page":0,"params":"query=LLM+benchmarks+are+misleading&advancedSyntax=true&analyticsTags=backend","processingTimeMS":34,"processingTimingsMS":{"_request":{"roundTrip":25},"afterFetch":{"format":{"highlighting":2,"total":2},"merge":{"mergeLoop":{"prepareNextHit":4,"total":4},"total":4},"total":4},"fetch":{"query":18,"scanning":9,"total":28},"total":34},"query":"LLM benchmarks are misleading","serverTimeMS":37}