{"exhaustive":{"nbHits":false,"typo":false},"exhaustiveNbHits":false,"exhaustiveTypo":false,"hits":[{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dhorthy"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["slop","code","bench"],"value":"Quick update here since a lot of folks asked about what slopcodebench is / where it fits in<p><em>Slop code bench</em> is unique among benchmarks in that it makes a model work against a progressively disclosed list of features (checkpoints) that it doesn\u2019t know it advance, inheriting the same codebase through the lifecycle of a single multi-checkpoint challenge<p>I like this because it much more mirrors how real software dev happens, rather than most benchmarks with one off tasks where slop code doesn\u2019t hinder future progress<p>The original post[1] and the SOL/fable follow up[2] went into a bit more detail on the benchmark and its challenges<p>1. <a href=\"https://news.ycombinator.com/item?id=49076391\">https://news.ycombinator.com/item?id=49076391</a><p>2. <a href=\"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-sol-fable-kimi-on-slop-code-bench.md\" rel=\"nofollow\">https://github.com/humanlayer/advanced-context-engineering-f...</a>"},"title":{"matchLevel":"none","matchedWords":[],"value":"SlopCodeBench 10/2026 \u2013 Opus 5.5, Astra wins overall, xhigh considered harmful"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["slop","code","bench"],"value":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-new-models-on-<em>slop-code-bench</em>.md"}},"_tags":["story","author_dhorthy","story_50108717"],"author":"dhorthy","children":[50171101,50171109,50171161,50171243,50171372,50171469,50171623,50171723,50171776,50172267,50172275,50172282,50172299,50172517,50172544,50172588,50172745,50173163,50173180,50173236,50173638,50173650,50173722,50173748,50174011,50174972,50175171,50175237,50175250,50175256,50177759,50177780,50180327,50180980,50205544],"created_at":"2026-10-15T19:09:34Z","created_at_i":1792091374,"num_comments":122,"objectID":"50108717","points":309,"story_id":50108717,"story_text":"Quick update here since a lot of folks asked about what slopcodebench is &#x2F; where it fits in<p>Slop code bench is unique among benchmarks in that it makes a model work against a progressively disclosed list of features (checkpoints) that it doesn\u2019t know it advance, inheriting the same codebase through the lifecycle of a single multi-checkpoint challenge<p>I like this because it much more mirrors how real software dev happens, rather than most benchmarks with one off tasks where slop code doesn\u2019t hinder future progress<p>The original post[1] and the SOL&#x2F;fable follow up[2] went into a bit more detail on the benchmark and its challenges<p>1. <a href=\"https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49076391\">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49076391</a><p>2. <a href=\"https:&#x2F;&#x2F;github.com&#x2F;humanlayer&#x2F;advanced-context-engineering-for-coding-agents&#x2F;blob&#x2F;main&#x2F;benchmarking-sol-fable-kimi-on-slop-code-bench.md\" rel=\"nofollow\">https:&#x2F;&#x2F;github.com&#x2F;humanlayer&#x2F;advanced-context-engineering-f...</a>","title":"SlopCodeBench 10/2026 \u2013 Opus 5.5, Astra wins overall, xhigh considered harmful","updated_at":"2026-11-16T01:33:13Z","url":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-new-models-on-slop-code-bench.md"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"eden-u4"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["slop","code","bench"],"value":"How is it different from <em>SlopCode</em> Bench? Isn't this a change in spec after the first prompt as in <em>SlopCode</em> Bench?"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"MaxxBench \u2013 \"Nice DeepSWE score. But is it true?\""},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://maxxbench.unlap.ai/research/why-maxxbench/"}},"_tags":["comment","author_eden-u4","story_50406222"],"author":"eden-u4","children":[50418780],"comment_text":"How is it different from SlopCode Bench? Isn&#x27;t this a change in spec after the first prompt as in SlopCode Bench?","created_at":"2026-11-08T11:37:16Z","created_at_i":1794137836,"objectID":"50412381","parent_id":50406222,"story_id":50406222,"story_title":"MaxxBench \u2013 \"Nice DeepSWE score. But is it true?\"","story_url":"https://maxxbench.unlap.ai/research/why-maxxbench/","updated_at":"2026-11-08T23:25:49Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"antipaul"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["slop","code","bench"],"value":"This is true. I was unfamiliar with this concept (I use codex/claude daily), and am trying to get to the &quot;bottom line&quot;.<p>Oh, it's there at bottom under &quot;my takeaway&quot;. But I still didn't get it initially.<p><em>Slop Code Bench</em>? Need a Slop Writing Bench too"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"SlopCodeBench 10/2026 \u2013 Opus 5.5, Astra wins overall, xhigh considered harmful"},"story_url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["slop","code","bench"],"value":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-new-models-on-<em>slop-code-bench</em>.md"}},"_tags":["comment","author_antipaul","story_50108717"],"author":"antipaul","comment_text":"This is true. I was unfamiliar with this concept (I use codex&#x2F;claude daily), and am trying to get to the &quot;bottom line&quot;.<p>Oh, it&#x27;s there at bottom under &quot;my takeaway&quot;. But I still didn&#x27;t get it initially.<p>Slop Code Bench? Need a Slop Writing Bench too","created_at":"2026-10-20T20:21:32Z","created_at_i":1792527692,"objectID":"50172107","parent_id":50171469,"story_id":50108717,"story_title":"SlopCodeBench 10/2026 \u2013 Opus 5.5, Astra wins overall, xhigh considered harmful","story_url":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-new-models-on-slop-code-bench.md","updated_at":"2026-10-20T21:47:03Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dhorthy"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["slop","code","bench"],"value":"my issue with frontier code is that it uses a model judge for quality whereas <em>slop code bench</em> forces a model to grapple with its own garbage code in order to receive a functionality reward"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Benchmarking Opus 5 on SlopCodeBench"},"story_url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["slop","code","bench"],"value":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-opus-5-on-<em>slop-code-bench</em>.md"}},"_tags":["comment","author_dhorthy","story_49076391"],"author":"dhorthy","comment_text":"my issue with frontier code is that it uses a model judge for quality whereas slop code bench forces a model to grapple with its own garbage code in order to receive a functionality reward","created_at":"2026-07-28T02:26:01Z","created_at_i":1785205561,"objectID":"49078496","parent_id":49078461,"story_id":49076391,"story_title":"Benchmarking Opus 5 on SlopCodeBench","story_url":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-opus-5-on-slop-code-bench.md","updated_at":"2026-07-28T11:11:35Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dhorthy"},"title":{"matchLevel":"none","matchedWords":[],"value":"Benchmarking Opus 5 on SlopCodeBench"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["slop","code","bench"],"value":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-opus-5-on-<em>slop-code-bench</em>.md"}},"_tags":["story","author_dhorthy","story_49076391"],"author":"dhorthy","children":[49077082,49077109,49077360,49077555,49077623,49077756,49077839,49077955,49078339,49078361,49078611,49078655,49079220,49079235,49079343,49079484,49079591,49079659,49080449,49080589,49080887,49081084,49081147,49081546,49081729,49081952,49082373,49082485,49082549,49082705,49083109,49084073,49084402,49084531,49085333,49088270,49089309],"created_at":"2026-07-27T22:37:52Z","created_at_i":1785191872,"num_comments":119,"objectID":"49076391","points":405,"story_id":49076391,"title":"Benchmarking Opus 5 on SlopCodeBench","updated_at":"2026-11-04T09:42:53Z","url":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-opus-5-on-slop-code-bench.md"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dhorthy"},"title":{"matchLevel":"none","matchedWords":[],"value":"Benchmarking Fable, Sol, and Kimi K3 on SlopCodeBench"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["slop","code","bench"],"value":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-sol-fable-kimi-on-<em>slop-code-bench</em>.md"}},"_tags":["story","author_dhorthy","story_49170037"],"author":"dhorthy","children":[49173001],"created_at":"2026-08-04T15:08:01Z","created_at_i":1785856081,"num_comments":1,"objectID":"49170037","points":8,"story_id":49170037,"title":"Benchmarking Fable, Sol, and Kimi K3 on SlopCodeBench","updated_at":"2026-08-04T22:15:17Z","url":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-sol-fable-kimi-on-slop-code-bench.md"}],"hitsPerPage":8,"nbHits":6,"nbPages":1,"page":0,"params":"query=slop-code-bench&hitsPerPage=8&advancedSyntax=true&analyticsTags=backend","processingTimeMS":25,"processingTimingsMS":{"_request":{"roundTrip":20},"fetch":{"query":5,"scanning":19,"total":25},"total":25},"query":"slop-code-bench","serverTimeMS":26}
