I've been playing with pi and the remote webui extension. I don't love it; it has a lot of chrome that obscures what I want to do. I just want a simple way to review the progress so far, and keep tweaking with minimal setup. Then, jump back into tmux when I'm back on my computer.
Thanks for your comments."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_xrd","story_49409092"],"author":"xrd","comment_text":"I basically do this, tmux via tailscale. But, I often get inspiration and want to jump into a webapp from my phone, or review.
I've been playing with pi and the remote webui extension. I don't love it; it has a lot of chrome that obscures what I want to do. I just want a simple way to review the progress so far, and keep tweaking with minimal setup. Then, jump back into tmux when I'm back on my computer.
Thanks for your comments.","created_at":"2026-08-26T00:24:53Z","created_at_i":1787703893,"objectID":"49442633","parent_id":49420163,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-26T00:30:34Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"pelasaco"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"> Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask.
Hm, but thats ok right? I mean some people like to code with LLM and other people like to let LLM code for them.. no?"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_pelasaco","story_49409092"],"author":"pelasaco","comment_text":"> Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask.
Hm, but thats ok right? I mean some people like to code with LLM and other people like to let LLM code for them.. no?","created_at":"2026-08-25T09:39:59Z","created_at_i":1787650799,"objectID":"49431163","parent_id":49411161,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-25T09:44:02Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"qarl2"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"OH - I think I understand now. You're like LeCun - dismissive of LLMs in all their forms. So of course agents can't lead to AGI, because LLMs just can't.
Which I've always found so strange. Language was always considered the pinnacle of the human mind - right up until we created LLMs. Now it's the world model - the things that animals always had and the thing we used to look down on.
From my perspective, language is still at the top. I'm not a fickle lover of the gaps."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_qarl2","story_49409092"],"author":"qarl2","comment_text":"OH - I think I understand now. You're like LeCun - dismissive of LLMs in all their forms. So of course agents can't lead to AGI, because LLMs just can't.
Which I've always found so strange. Language was always considered the pinnacle of the human mind - right up until we created LLMs. Now it's the world model - the things that animals always had and the thing we used to look down on.
From my perspective, language is still at the top. I'm not a fickle lover of the gaps.","created_at":"2026-08-25T09:14:06Z","created_at_i":1787649246,"objectID":"49430998","parent_id":49421617,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-25T09:17:47Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mijowi"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"Although obviously unserious, I would argue a baby harness is mostly the opposite of what an agent harness does. A baby harness constrains and contains the baby, while an agent harness controls but also empowers the agent."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_mijowi","story_49409092"],"author":"mijowi","comment_text":"Although obviously unserious, I would argue a baby harness is mostly the opposite of what an agent harness does. A baby harness constrains and contains the baby, while an agent harness controls but also empowers the agent.","created_at":"2026-08-25T06:48:49Z","created_at_i":1787640529,"objectID":"49429949","parent_id":49414083,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-25T06:51:31Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"stymaar"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"> then you should be able to add features by just reading the acceptance test and scanning unit tests.
That \u201cjust\u201d is bearing a lot of weight though as tests are often even longer than the code itself, in addition to being excruciating to review."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_stymaar","story_49409092"],"author":"stymaar","comment_text":"> then you should be able to add features by just reading the acceptance test and scanning unit tests.
That \u201cjust\u201d is bearing a lot of weight though as tests are often even longer than the code itself, in addition to being excruciating to review.","created_at":"2026-08-24T21:54:50Z","created_at_i":1787608490,"objectID":"49426351","parent_id":49420578,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-25T14:11:49Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"sysfiend"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"Model -> brain cells' conections\nHarness -> everything else
I've been working with different setups in parallel for months (openclaw, pi, cursor per project harnesses and codex) and, even though using the same model most of the time, I can clearly see how the behave in very different ways deppending on the setup.
As a language model, language is our way to communicate and build everything around the models, which makes my younger self (who loved writing stories) very very happy :)"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_sysfiend","story_49409092"],"author":"sysfiend","comment_text":"Model -> brain cells' conections\nHarness -> everything else
I've been working with different setups in parallel for months (openclaw, pi, cursor per project harnesses and codex) and, even though using the same model most of the time, I can clearly see how the behave in very different ways deppending on the setup.
As a language model, language is our way to communicate and build everything around the models, which makes my younger self (who loved writing stories) very very happy :)","created_at":"2026-08-24T21:20:24Z","created_at_i":1787606424,"objectID":"49425993","parent_id":49409092,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-24T21:24:01Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ni10c"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"Thanks for these thoughts. Super insightful and appreciated"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_ni10c","story_49409092"],"author":"ni10c","comment_text":"Thanks for these thoughts. Super insightful and appreciated","created_at":"2026-08-24T16:16:39Z","created_at_i":1787588199,"objectID":"49421975","parent_id":49411715,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-24T19:43:47Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"okamiueru"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"I find the vocabulary used to be disturbingly fascinating, to the degree concepts are being anthropomorphized. It's so pervasive that I cannot help but assume it is entirely deliberate.
The principles by which LLMs functions haven't changed in the last four years. It is still a next-word-predictor, a statistical parrot, if you will. But if you don't understand the mechanisms behind it, you cannot be faulted for thinking this is something much more. Most of it, is pretty devious marketing.
As an example, no LLM model does anything that can be considered "reasoning", or "intelligent" in the traditional sense, but these words are used extensively. "High reasoning model" is a pricing tier. The article in question has an anthropomorphized term in every single sentence. I'll pick a paragraph at random and highlight the cases. If both parties understand the mechanisms, these words are fine, and we do that all that time. The issue is when one sides is mislead to believe that these systems can be relied on in a way that they should not be, leading to people getting hurt.
> > The *translation layer* is what *allows* a *harness* to *work* with different AI models. In some cases, a *harness may decide* to *use* different models within the same *agentic loop*, because different AI *models may excel* at different tasks. The *translation layer* is also a crucial aspect of *harnesses* because they *deliver control* to the end user. It means that someone can take their *AI harness* and use it with a model from Anthropic, or OpenAI, or explore one of the open weight AI models that often deliver great value-for-money (measured by cost-per-task).
The underlying logic isn't remotely as mysterious or mystic as the language makes it seem. A different paragraph:
> > Tools are a set of *capabilities*, written in code, that the model can *\u201ccall\u201d*. The *harness describes* the tools and also *provides* the software that is the tool itself. Examples of these tools might include a web search tool, a tool that *allows the model* to write and execute software code, or a tool that *allows the model* to *compose* an email. Critically, the *harness usually* does not *dictate* when and how the *AI* model should *use* the tool. Instead, it simply *makes* the tools available, *describes* them clearly, and *allows* the *AI* model itself to *decide* when and how *it should use* them"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_okamiueru","story_49409092"],"author":"okamiueru","comment_text":"I find the vocabulary used to be disturbingly fascinating, to the degree concepts are being anthropomorphized. It's so pervasive that I cannot help but assume it is entirely deliberate.
The principles by which LLMs functions haven't changed in the last four years. It is still a next-word-predictor, a statistical parrot, if you will. But if you don't understand the mechanisms behind it, you cannot be faulted for thinking this is something much more. Most of it, is pretty devious marketing.
As an example, no LLM model does anything that can be considered "reasoning", or "intelligent" in the traditional sense, but these words are used extensively. "High reasoning model" is a pricing tier. The article in question has an anthropomorphized term in every single sentence. I'll pick a paragraph at random and highlight the cases. If both parties understand the mechanisms, these words are fine, and we do that all that time. The issue is when one sides is mislead to believe that these systems can be relied on in a way that they should not be, leading to people getting hurt.
> > The *translation layer* is what *allows* a *harness* to *work* with different AI models. In some cases, a *harness may decide* to *use* different models within the same *agentic loop*, because different AI *models may excel* at different tasks. The *translation layer* is also a crucial aspect of *harnesses* because they *deliver control* to the end user. It means that someone can take their *AI harness* and use it with a model from Anthropic, or OpenAI, or explore one of the open weight AI models that often deliver great value-for-money (measured by cost-per-task).
The underlying logic isn't remotely as mysterious or mystic as the language makes it seem. A different paragraph:
> > Tools are a set of *capabilities*, written in code, that the model can *\u201ccall\u201d*. The *harness describes* the tools and also *provides* the software that is the tool itself. Examples of these tools might include a web search tool, a tool that *allows the model* to write and execute software code, or a tool that *allows the model* to *compose* an email. Critically, the *harness usually* does not *dictate* when and how the *AI* model should *use* the tool. Instead, it simply *makes* the tools available, *describes* them clearly, and *allows* the *AI* model itself to *decide* when and how *it should use* them","created_at":"2026-08-24T16:12:41Z","created_at_i":1787587961,"objectID":"49421899","parent_id":49409953,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-31T16:07:08Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"grey-area"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"I\u2019m pleased for you.
In my experience the use of agents and harnesses and other scaffolding around them doesn\u2019t improve the performance of LLMs much, which is adequate for some tasks under supervision but nothing like general intelligence (or electricity for that matter). But good that it works for you."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_grey-area","story_49409092"],"author":"grey-area","children":[49421784,49430998],"comment_text":"I\u2019m pleased for you.
In my experience the use of agents and harnesses and other scaffolding around them doesn\u2019t improve the performance of LLMs much, which is adequate for some tasks under supervision but nothing like general intelligence (or electricity for that matter). But good that it works for you.","created_at":"2026-08-24T15:56:48Z","created_at_i":1787587008,"objectID":"49421617","parent_id":49418488,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-27T09:16:09Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"flumes_whims_"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"MCP clutters the context. Much better to have cli that has --help on subcommands so it can get the parts in needs in the current context."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_flumes_whims_","story_49409092"],"author":"flumes_whims_","children":[49472157],"comment_text":"MCP clutters the context. Much better to have cli that has --help on subcommands so it can get the parts in needs in the current context.","created_at":"2026-08-24T15:50:02Z","created_at_i":1787586602,"objectID":"49421506","parent_id":49419571,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-27T22:34:41Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"theptip"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"The trick is to set up the harness so that the solution is easy to verify - you\u2019ve profited as long as verification is cheaper than building, but ideally verification is close to automatic (not always achievable of course).
Generally you want to include objective/repeatable outputs as citations.
An example would be, if you invest in an awesome layered test rig (browser test, fuzz/property tests, very well reviewed unit/integration tests, etc.) then you should be able to add features by just reading the acceptance test and scanning unit tests."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_theptip","story_49409092"],"author":"theptip","children":[49426351],"comment_text":"The trick is to set up the harness so that the solution is easy to verify - you\u2019ve profited as long as verification is cheaper than building, but ideally verification is close to automatic (not always achievable of course).
Generally you want to include objective/repeatable outputs as citations.
An example would be, if you invest in an awesome layered test rig (browser test, fuzz/property tests, very well reviewed unit/integration tests, etc.) then you should be able to add features by just reading the acceptance test and scanning unit tests.","created_at":"2026-08-24T14:47:55Z","created_at_i":1787582875,"objectID":"49420578","parent_id":49416107,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-26T03:29:34Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"gf000"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"Not really solving all your cases, but I found tmux (or herdr) on a home server works pretty well. I just ssh into my home server and continue where I left off with the same claude/opencode open.
On a longer term, I think "assistant" style harnesses might help here, like vellum.ai. I no longer use that, but I asked it to create an ACP proxy through iroh (basically tailscale but on the application layer), and it managed to control claude on another device of mine. A friend did similar stuff with tailscale.
I have started writing a hobby harness with a web interface where I would like to support this "ACP proxy" mode natively, and also to make the models aware of different devices in some way and "move" work between them."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_gf000","story_49409092"],"author":"gf000","children":[49442633],"comment_text":"Not really solving all your cases, but I found tmux (or herdr) on a home server works pretty well. I just ssh into my home server and continue where I left off with the same claude/opencode open.
On a longer term, I think "assistant" style harnesses might help here, like vellum.ai. I no longer use that, but I asked it to create an ACP proxy through iroh (basically tailscale but on the application layer), and it managed to control claude on another device of mine. A friend did similar stuff with tailscale.
I have started writing a hobby harness with a web interface where I would like to support this "ACP proxy" mode natively, and also to make the models aware of different devices in some way and "move" work between them.","created_at":"2026-08-24T14:18:23Z","created_at_i":1787581103,"objectID":"49420163","parent_id":49410304,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-26T00:25:49Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"visarga"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"> If you want to follow a process or a checklist you probably shouldn't use an LLM
I like to externalize tasks as markdown files with checklists, they are still planned by agents but I can pass the plan around to judge agents and fix some errors before implementing.
I also have the coding agents comment on each closed checklist item, so the same file becomes a log of what happened. This goes to the implementation judge. I can also switch agents anytime, or resume a task days later no problem.
I am avoiding internally provided tools for todo lists and planning because they do not leave the same artifact trail which makes judging with separate agents easy."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_visarga","story_49409092"],"author":"visarga","comment_text":"> If you want to follow a process or a checklist you probably shouldn't use an LLM
I like to externalize tasks as markdown files with checklists, they are still planned by agents but I can pass the plan around to judge agents and fix some errors before implementing.
I also have the coding agents comment on each closed checklist item, so the same file becomes a log of what happened. This goes to the implementation judge. I can also switch agents anytime, or resume a task days later no problem.
I am avoiding internally provided tools for todo lists and planning because they do not leave the same artifact trail which makes judging with separate agents easy.","created_at":"2026-08-24T06:34:24Z","created_at_i":1787553264,"objectID":"49415914","parent_id":49411161,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-26T03:28:50Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dbmikus"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"For a truly "handoff-able" session, you also need to store the data it was working on. In most cases, these are the git patches"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_dbmikus","story_49409092"],"author":"dbmikus","comment_text":"For a truly "handoff-able" session, you also need to store the data it was working on. In most cases, these are the git patches","created_at":"2026-08-24T04:20:35Z","created_at_i":1787545235,"objectID":"49415145","parent_id":49411307,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-25T12:12:33Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"johnmw"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"I wonder if I'm the only person whose first analogy that came to mind was: model = toddler, harness = baby bouncing jumper harness"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_johnmw","story_49409092"],"author":"johnmw","children":[49429949],"comment_text":"I wonder if I'm the only person whose first analogy that came to mind was: model = toddler, harness = baby bouncing jumper harness","created_at":"2026-08-24T00:49:47Z","created_at_i":1787532587,"objectID":"49414083","parent_id":49411701,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-25T06:49:31Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"rush86999"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"Reasoning / self-consistency (voter in core/llm/self_consistency_voter.py):\n- Wang et al. Self-Consistency Improves Chain-of-Thought \u2014 ICLR 2023, Google Brain, 4k+ cites \u2014 https://arxiv.org/abs/2203.11171 \u2014 N-sample majority vote we use verbatim\n- Chen et al. Universal Self-Consistency \u2014 ICML 2024 \u2014 https://arxiv.org/abs/2311.17311 \u2014 judge fallback when no hash collides\n- Soft Self-Consistency \u2014 ACL 2024 \u2014 https://aclanthology.org/2024.acl-short.28.pdf\n- Too Consistent to Detect \u2014 EMNLP 2025 \u2014 https://aclanthology.org/2025.emnlp-main.238/ \u2014 why SC doesn't fix systematic bias\n- Self-Consistency Falls Short \u2014 TACL \u2014 https://direct.mit.org/tacl/article/doi/10.1162/TACL.a.625/ \u2014 position-bias failure mode
Multi-agent / org (core/agent_radio/, core/fleet_orchestration/):\n- Stanford Virtual Biotech \u2014 bioRxiv 2026.02.23.707551, Zou Lab \u2014 https://www.biorxiv.org/content/10.64898/2026.02.23.707551v1 \u2014 37k agents, CSO->scientists->reviewer->re-delegation, Merck external validation of B7-H3 design. Basis for VFS + hierarchy.\n- Debate or Vote (Choi & Li) \u2014 NeurIPS 2025 \u2014 https://arxiv.org/abs/2508.17536 \u2014 MAD gains = majority vote, not debate (why we didn't build debate)
Sandbox / eval:\n- DABstep \u2014 arXiv:2506.23719 \u2014 https://arxiv.org/abs/2506.23719 \u2014 450 real Adyen tasks, justifies code-interpreter + sandbox isolation\n- Spotlighting \u2014 Microsoft Research \u2014 https://arxiv.org/abs/2403.14720 \u2014 provenance delimiters cut injection ASR 50% -> <2%
Governance:\n- OWASP Top 10 for Agentic Applications 2026 \u2014 globally peer-reviewed by 100+ experts, Dec 2025 \u2014 https://genai.owasp.org/resource/owasp-top-10-for-agentic-ap... \u2014 HIGH. Atom maps 1:1 (Goal Hijack -> match-confidence + oracle, Tool Misuse -> sandbox whitelist/caps, Privilege Abuse -> capability bindings, Memory Poisoning -> verified-episode graduation, etc.) docs/marketing/RESEARCH_NOTES.md:130\n- NIST AI Agent Standards Initiative \u2014 Feb 17 2026, NIST CAISI \u2014 https://www.nist.gov/artificial-intelligence/ai-agent-standa... + RFI summary May 2026 https://www.nist.gov/publications/summary-analysis-responses... \u2014 HIGH (US gov standard). Defines the 4 enterprise minimums Atom implements: identification, authorization, access delegation, logging.\n- Stanford Virtual Biotech \u2014 bioRxiv 2026.02.23.707551 \u2014 https://www.biorxiv.org/content/10.64898/2026.02.23.707551v1 \u2014 CSO -> 4 divisions -> 8 scientists -> reviewer -> re-delegation, no debate, no SFT \u2014 HIGH (Stanford Zou lab + Merck external validation). Basis for Atom's fleet hierarchy core/agent_radio/ and why maturity is routing not security.\n- Spotlighting \u2014 Microsoft Research \u2014 https://arxiv.org/abs/2403.14720 \u2014 HIGH \u2014 provenance delimiters <provenance type="tool_output"> cut indirect injection ASR 50% -> <2%, used in core/provenance.py:10\n- IntentGuard \u2014 https://arxiv.org/abs/2512.00966 + OpenReview \u2014 HIGH \u2014 intent tracing ASR 100% -> 8.5% on AgentDojo/Mind2Web, basis for sandbox egress allowlist + core/sandbox_tripwire.py"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_rush86999","story_49409092"],"author":"rush86999","comment_text":"Reasoning / self-consistency (voter in core/llm/self_consistency_voter.py):\n- Wang et al. Self-Consistency Improves Chain-of-Thought \u2014 ICLR 2023, Google Brain, 4k+ cites \u2014 https://arxiv.org/abs/2203.11171 \u2014 N-sample majority vote we use verbatim\n- Chen et al. Universal Self-Consistency \u2014 ICML 2024 \u2014 https://arxiv.org/abs/2311.17311 \u2014 judge fallback when no hash collides\n- Soft Self-Consistency \u2014 ACL 2024 \u2014 https://aclanthology.org/2024.acl-short.28.pdf\n- Too Consistent to Detect \u2014 EMNLP 2025 \u2014 https://aclanthology.org/2025.emnlp-main.238/ \u2014 why SC doesn't fix systematic bias\n- Self-Consistency Falls Short \u2014 TACL \u2014 https://direct.mit.org/tacl/article/doi/10.1162/TACL.a.625/ \u2014 position-bias failure mode
Multi-agent / org (core/agent_radio/, core/fleet_orchestration/):\n- Stanford Virtual Biotech \u2014 bioRxiv 2026.02.23.707551, Zou Lab \u2014 https://www.biorxiv.org/content/10.64898/2026.02.23.707551v1 \u2014 37k agents, CSO->scientists->reviewer->re-delegation, Merck external validation of B7-H3 design. Basis for VFS + hierarchy.\n- Debate or Vote (Choi & Li) \u2014 NeurIPS 2025 \u2014 https://arxiv.org/abs/2508.17536 \u2014 MAD gains = majority vote, not debate (why we didn't build debate)
Sandbox / eval:\n- DABstep \u2014 arXiv:2506.23719 \u2014 https://arxiv.org/abs/2506.23719 \u2014 450 real Adyen tasks, justifies code-interpreter + sandbox isolation\n- Spotlighting \u2014 Microsoft Research \u2014 https://arxiv.org/abs/2403.14720 \u2014 provenance delimiters cut injection ASR 50% -> <2%
Governance:\n- OWASP Top 10 for Agentic Applications 2026 \u2014 globally peer-reviewed by 100+ experts, Dec 2025 \u2014 https://genai.owasp.org/resource/owasp-top-10-for-agentic-ap... \u2014 HIGH. Atom maps 1:1 (Goal Hijack -> match-confidence + oracle, Tool Misuse -> sandbox whitelist/caps, Privilege Abuse -> capability bindings, Memory Poisoning -> verified-episode graduation, etc.) docs/marketing/RESEARCH_NOTES.md:130\n- NIST AI Agent Standards Initiative \u2014 Feb 17 2026, NIST CAISI \u2014 https://www.nist.gov/artificial-intelligence/ai-agent-standa... + RFI summary May 2026 https://www.nist.gov/publications/summary-analysis-responses... \u2014 HIGH (US gov standard). Defines the 4 enterprise minimums Atom implements: identification, authorization, access delegation, logging.\n- Stanford Virtual Biotech \u2014 bioRxiv 2026.02.23.707551 \u2014 https://www.biorxiv.org/content/10.64898/2026.02.23.707551v1 \u2014 CSO -> 4 divisions -> 8 scientists -> reviewer -> re-delegation, no debate, no SFT \u2014 HIGH (Stanford Zou lab + Merck external validation). Basis for Atom's fleet hierarchy core/agent_radio/ and why maturity is routing not security.\n- Spotlighting \u2014 Microsoft Research \u2014 https://arxiv.org/abs/2403.14720 \u2014 HIGH \u2014 provenance delimiters <provenance type="tool_output"> cut indirect injection ASR 50% -> <2%, used in core/provenance.py:10\n- IntentGuard \u2014 https://arxiv.org/abs/2512.00966 + OpenReview \u2014 HIGH \u2014 intent tracing ASR 100% -> 8.5% on AgentDojo/Mind2Web, basis for sandbox egress allowlist + core/sandbox_tripwire.py","created_at":"2026-08-23T23:38:54Z","created_at_i":1787528334,"objectID":"49413669","parent_id":49413598,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-09-16T19:46:00Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"polotics"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":"Hi. This is very interesting, could you link to the research? There is a dearth of proper research studies that A/B test what approach is best in terms of harness structure based on repeatable benchmark data with relevant sample uses-cases."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What Is a Harness?"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://earendil.com/posts/what-is-a-harness/"}},"_tags":["comment","author_polotics","story_49409092"],"author":"polotics","children":[49413669],"comment_text":"Hi. This is very interesting, could you link to the research? There is a dearth of proper research studies that A/B test what approach is best in terms of harness structure based on repeatable benchmark data with relevant sample uses-cases.","created_at":"2026-08-23T23:24:07Z","created_at_i":1787527447,"objectID":"49413598","parent_id":49413037,"story_id":49409092,"story_title":"What Is a Harness?","story_url":"https://earendil.com/posts/what-is-a-harness/","updated_at":"2026-08-29T04:01:44Z"}],"hitsPerPage":20,"nbHits":55,"nbPages":3,"page":0,"params":"tags=story_49409092&advancedSyntax=true&analyticsTags=backend","processingTimeMS":4,"processingTimingsMS":{"_request":{"roundTrip":19},"afterFetch":{"format":{"highlighting":1,"total":1}},"getIdx":{"load":{"gens":2,"total":3},"total":3},"total":4},"query":"","serverTimeMS":7}