1. You can use the '/new-tool' and tell what kind of tool you want (including whether it should be task-scoped, workspace-scoped, or global), the model builds it, the harness runs validation and other tests until the tool is ready
2. The model decides that in such and such task, it would be helpful to have a tool like this, it can build a task-scoped tool.
In either scenario, the tool catalog is rebuilt, and the new tool is instantly available in the next turn.","title":null,"type":"comment","url":null},{"author":"timbowhite","children":[],"created_at":"2026-08-23T15:14:20.000Z","created_at_i":1787498060,"id":49409482,"options":[],"parent_id":49409371,"points":null,"story_id":49409092,"text":"Pi's most popular extensions, by download count:
https://pi.dev/packages?type=extension","title":null,"type":"comment","url":null},{"author":"dominotw","children":[],"created_at":"2026-08-23T15:14:47.000Z","created_at_i":1787498087,"id":49409486,"options":[],"parent_id":49409371,"points":null,"story_id":49409092,"text":"i think its the opposite. claude code apparently removed hundreds of lines of system prompt because its not relavent anymore with newer models.
also i think its hard to build general harnesses if they were trained on specific harness architecture.","title":null,"type":"comment","url":null},{"author":"mpawelski","children":[{"author":"grey-area","children":[],"created_at":"2026-08-23T15:28:51.000Z","created_at_i":1787498931,"id":49409604,"options":[],"parent_id":49409493,"points":null,"story_id":49409092,"text":"Sadly, many people have bought into the cult that LLMs will lead to AGI. I guess if that is your worldview then all this babbling about new frontiers makes more sense.
They probably used an LLM to come up with this bizarre metaphor.","title":null,"type":"comment","url":null},{"author":"wwalexander","children":[{"author":"tomrod","children":[],"created_at":"2026-08-23T20:47:28.000Z","created_at_i":1787518048,"id":49412462,"options":[],"parent_id":49409928,"points":null,"story_id":49409092,"text":"If we take this at face value, this means AI = 0 !","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T16:06:24.000Z","created_at_i":1787501184,"id":49409928,"options":[],"parent_id":49409493,"points":null,"story_id":49409092,"text":"E = mc^2 + AI","title":null,"type":"comment","url":null},{"author":"DarmokTanagra","children":[],"created_at":"2026-08-23T17:59:43.000Z","created_at_i":1787507983,"id":49410930,"options":[],"parent_id":49409493,"points":null,"story_id":49409092,"text":"Its literally the same people who were making hyperbolic crypto claims a few years ago.
This entire forum is infested with shameless hype chasers and biological linkedin bots.","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T15:15:39.000Z","created_at_i":1787498139,"id":49409493,"options":[],"parent_id":49409371,"points":null,"story_id":49409092,"text":"> Harnesses are the next frontier. If LLMs are electricity, harnesses are the \u201celectronics.\u201d
I really though this comment was a satire ...","title":null,"type":"comment","url":null},{"author":"sejje","children":[],"created_at":"2026-08-23T15:17:01.000Z","created_at_i":1787498221,"id":49409509,"options":[],"parent_id":49409371,"points":null,"story_id":49409092,"text":"What did you bring over from prime-agent? (I use prime-agent as my daily since it launched)
I primarily like how it manages sessions, and how agents can easily reference other sessions.","title":null,"type":"comment","url":null},{"author":"amelius","children":[{"author":"conmod278","children":[{"author":"layer8","children":[],"created_at":"2026-08-23T16:40:12.000Z","created_at_i":1787503212,"id":49410225,"options":[],"parent_id":49409632,"points":null,"story_id":49409092,"text":"I\u2019d say that harnesses almost by definition are the parts that you want to keep customizable. That won\u2019t get absorbed into the weights.","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T15:31:03.000Z","created_at_i":1787499063,"id":49409632,"options":[],"parent_id":49409573,"points":null,"story_id":49409092,"text":"https://www.latent.space/p/attention-interface
Labs are now post-training models with Harness so that Harness now gets absorbed into the weights.","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T15:25:47.000Z","created_at_i":1787498747,"id":49409573,"options":[],"parent_id":49409371,"points":null,"story_id":49409092,"text":"> If LLMs are electricity, harnesses are the \u201celectronics.\u201d (...) the harnesses will be the actual value providers.
Don't get ahead of yourself. Harnesses are not exactly rocket science and will be a commodity.
The real value providers here are the hardware, then the LLM as a distant second, and at a much larger distance the harness.","title":null,"type":"comment","url":null},{"author":"grim_io","children":[],"created_at":"2026-08-23T15:54:25.000Z","created_at_i":1787500465,"id":49409817,"options":[],"parent_id":49409371,"points":null,"story_id":49409092,"text":"I don't think so.
What I can see is a world where we end up with a Chromium-shaped harness, a fully featured standard implementation everyone builds against, because doing every single thing yourself would be crazy.
The antithesis to Pi, if you will.","title":null,"type":"comment","url":null},{"author":"dbrecht_","children":[],"created_at":"2026-08-27T22:37:17.000Z","created_at_i":1787870237,"id":49472184,"options":[],"parent_id":49409371,"points":null,"story_id":49409092,"text":"I've been getting a little frustrated with having to rearchitect things any time I want to try a new harness. Wrote about my most recent experiments with separating conversation from control loop here and using MCP as the seam here: https://demianbrecht.com/posts/the-harness-within-the-harnes.... This allows me to build a spectrum of agentic to entirely deterministic tools and be able to port them from one harness to another with only a minimal amount of harness-specific config.","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T15:00:34.000Z","created_at_i":1787497234,"id":49409371,"options":[],"parent_id":49409092,"points":null,"story_id":49409092,"text":"Harnesses are the next frontier. If LLMs are electricity, harnesses are the \u201celectronics.\u201d Right now, it\u2019s like an AC vs DC between Claude and ChatGPT, but once that settles, the harnesses will be the actual value providers.
And Pi is the best harness because of the amazing extension system. You can build extensions that turn Pi into a stock trader, software factory, anything. I tried switching to another harness but none have extension functionality as good as Pi.
Even if there is a new harness or agent project, I tell Pi to dig into the codebase and then make me an extension that brings that functionality into Pi. I did it with Prime Intellect\u2019s and Deepseek\u2019s harnesses and those are built on Pi.","title":null,"type":"comment","url":null},{"author":"jascha_eng","children":[],"created_at":"2026-08-23T15:02:53.000Z","created_at_i":1787497373,"id":49409388,"options":[],"parent_id":49409092,"points":null,"story_id":49409092,"text":"The ai hype word for 2026 after agent in 2025 for any LLM powered application.
Well kind of, I wouldn't be surprised to see that some things marketed as agents are actually good old deterministic software.","title":null,"type":"comment","url":null},{"author":"Syntaf","children":[{"author":"YZF","children":[{"author":"wonnage","children":[{"author":"0x457","children":[],"created_at":"2026-08-23T19:39:00.000Z","created_at_i":1787513940,"id":49411792,"options":[],"parent_id":49411302,"points":null,"story_id":49409092,"text":"> . \u201cwhy app slow\u201d obviously doesn\u2019t work because the task is underspecified.
Not always. In my case LLM goes to grafana mcp, pulls metrics/traces/cpu profiles. Figures out what is slow and proposes a solution.","title":null,"type":"comment","url":null},{"author":"pests","children":[],"created_at":"2026-08-23T23:20:21.000Z","created_at_i":1787527221,"id":49413574,"options":[],"parent_id":49411302,"points":null,"story_id":49409092,"text":"> \u201cwhy app slow\u201d obviously doesn\u2019t work because the task is underspecified
Definitely not true and like everyone else is saying, shows how people still underestimate these models.
I have been working on a simple vite + react app lately and commonly ask Gemini/Antigravity to just "improve speeds", "x is running slow, check it out" and have no complaints.","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T18:44:26.000Z","created_at_i":1787510666,"id":49411302,"options":[],"parent_id":49411161,"points":null,"story_id":49409092,"text":"That works for well trod paths, e.g \u201cfix ci\u201d works exceedingly well. \u201cwhy app slow\u201d obviously doesn\u2019t work because the task is underspecified. But in order to properly specify you either need an experienced engineer who knows how to narrow the problem domain, or you have to provide some template instructions/output formats (e.g, skills) which will invariably never fit the problem perfectly","title":null,"type":"comment","url":null},{"author":"visarga","children":[],"created_at":"2026-08-24T06:34:24.000Z","created_at_i":1787553264,"id":49415914,"options":[],"parent_id":49411161,"points":null,"story_id":49409092,"text":"> If you want to follow a process or a checklist you probably shouldn't use an LLM
I like to externalize tasks as markdown files with checklists, they are still planned by agents but I can pass the plan around to judge agents and fix some errors before implementing.
I also have the coding agents comment on each closed checklist item, so the same file becomes a log of what happened. This goes to the implementation judge. I can also switch agents anytime, or resume a task days later no problem.
I am avoiding internally provided tools for todo lists and planning because they do not leave the same artifact trail which makes judging with separate agents easy.","title":null,"type":"comment","url":null},{"author":"pelasaco","children":[],"created_at":"2026-08-25T09:39:59.000Z","created_at_i":1787650799,"id":49431163,"options":[],"parent_id":49411161,"points":null,"story_id":49409092,"text":"> Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask.
Hm, but thats ok right? I mean some people like to code with LLM and other people like to let LLM code for them.. no?","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T18:25:28.000Z","created_at_i":1787509528,"id":49411161,"options":[],"parent_id":49410048,"points":null,"story_id":49409092,"text":"This is the same "tension" I keep seeing in my day job. Some people approach LLMs like they're writing code. They give a long list of detailed instructions for specific scenarios. When I use LLMs I leave things as open as possible. I just give them the information they need and my ask.
As you say frontier models are very good at figuring things out. Being too prescriptive is counterproductive, it over-constrains the model, it fills the context with conflicting instructions, it reduces the ability of the agent to respond to novel situations (and really in real life most situations are going to be novel). If you want to follow a process or a checklist you probably shouldn't use an LLM, or you should use it for some sub-tasks in the checklist/process but something more deterministic to work through the list.","title":null,"type":"comment","url":null},{"author":"rush86999","children":[{"author":"polotics","children":[{"author":"rush86999","children":[],"created_at":"2026-08-23T23:38:54.000Z","created_at_i":1787528334,"id":49413669,"options":[],"parent_id":49413598,"points":null,"story_id":49409092,"text":"Reasoning / self-consistency (voter in core/llm/self_consistency_voter.py):\n- Wang et al. Self-Consistency Improves Chain-of-Thought \u2014 ICLR 2023, Google Brain, 4k+ cites \u2014 https://arxiv.org/abs/2203.11171 \u2014 N-sample majority vote we use verbatim\n- Chen et al. Universal Self-Consistency \u2014 ICML 2024 \u2014 https://arxiv.org/abs/2311.17311 \u2014 judge fallback when no hash collides\n- Soft Self-Consistency \u2014 ACL 2024 \u2014 https://aclanthology.org/2024.acl-short.28.pdf\n- Too Consistent to Detect \u2014 EMNLP 2025 \u2014 https://aclanthology.org/2025.emnlp-main.238/ \u2014 why SC doesn't fix systematic bias\n- Self-Consistency Falls Short \u2014 TACL \u2014 https://direct.mit.org/tacl/article/doi/10.1162/TACL.a.625/ \u2014 position-bias failure mode
Multi-agent / org (core/agent_radio/, core/fleet_orchestration/):\n- Stanford Virtual Biotech \u2014 bioRxiv 2026.02.23.707551, Zou Lab \u2014 https://www.biorxiv.org/content/10.64898/2026.02.23.707551v1 \u2014 37k agents, CSO->scientists->reviewer->re-delegation, Merck external validation of B7-H3 design. Basis for VFS + hierarchy.\n- Debate or Vote (Choi & Li) \u2014 NeurIPS 2025 \u2014 https://arxiv.org/abs/2508.17536 \u2014 MAD gains = majority vote, not debate (why we didn't build debate)
Sandbox / eval:\n- DABstep \u2014 arXiv:2506.23719 \u2014 https://arxiv.org/abs/2506.23719 \u2014 450 real Adyen tasks, justifies code-interpreter + sandbox isolation\n- Spotlighting \u2014 Microsoft Research \u2014 https://arxiv.org/abs/2403.14720 \u2014 provenance delimiters cut injection ASR 50% -> <2%
Governance:\n- OWASP Top 10 for Agentic Applications 2026 \u2014 globally peer-reviewed by 100+ experts, Dec 2025 \u2014 https://genai.owasp.org/resource/owasp-top-10-for-agentic-ap... \u2014 HIGH. Atom maps 1:1 (Goal Hijack -> match-confidence + oracle, Tool Misuse -> sandbox whitelist/caps, Privilege Abuse -> capability bindings, Memory Poisoning -> verified-episode graduation, etc.) docs/marketing/RESEARCH_NOTES.md:130\n- NIST AI Agent Standards Initiative \u2014 Feb 17 2026, NIST CAISI \u2014 https://www.nist.gov/artificial-intelligence/ai-agent-standa... + RFI summary May 2026 https://www.nist.gov/publications/summary-analysis-responses... \u2014 HIGH (US gov standard). Defines the 4 enterprise minimums Atom implements: identification, authorization, access delegation, logging.\n- Stanford Virtual Biotech \u2014 bioRxiv 2026.02.23.707551 \u2014 https://www.biorxiv.org/content/10.64898/2026.02.23.707551v1 \u2014 CSO -> 4 divisions -> 8 scientists -> reviewer -> re-delegation, no debate, no SFT \u2014 HIGH (Stanford Zou lab + Merck external validation). Basis for Atom's fleet hierarchy core/agent_radio/ and why maturity is routing not security.\n- Spotlighting \u2014 Microsoft Research \u2014 https://arxiv.org/abs/2403.14720 \u2014 HIGH \u2014 provenance delimiters <provenance type="tool_output"> cut indirect injection ASR 50% -> <2%, used in core/provenance.py:10\n- IntentGuard \u2014 https://arxiv.org/abs/2512.00966 + OpenReview \u2014 HIGH \u2014 intent tracing ASR 100% -> 8.5% on AgentDojo/Mind2Web, basis for sandbox egress allowlist + core/sandbox_tripwire.py","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T23:24:07.000Z","created_at_i":1787527447,"id":49413598,"options":[],"parent_id":49413037,"points":null,"story_id":49409092,"text":"Hi. This is very interesting, could you link to the research? There is a dearth of proper research studies that A/B test what approach is best in terms of harness structure based on repeatable benchmark data with relevant sample uses-cases.","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T21:59:45.000Z","created_at_i":1787522385,"id":49413037,"options":[],"parent_id":49410048,"points":null,"story_id":49409092,"text":"I think you've really hit the mark on how the harness should be structured:
1. Guardrails - deterministic, social intelligence, team alignment & accountability\n2. Learn by doing\n3. make it stupid easy for the agent to research and access data\n4. DRY
Research supports this. \nTry picking up some ideas from my harness: https://github.com/rush86999/atom","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T16:19:44.000Z","created_at_i":1787501984,"id":49410048,"options":[],"parent_id":49409092,"points":null,"story_id":49409092,"text":"I\u2019ve been working on a harness for accounting agents at my job recently and it\u2019s been a pretty interesting experience.
We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It\u2019s both fun to build and extremely useful for agents.
We paired this with skills initially, but found that the way folks built skills was often too prescriptive and limited to the authors own specific function in the company. A 2k line long skill suffers from the same gaps as we do, if an agent is just following a laundry list it\u2019s less likely to reason about the request it\u2019s doing.
So we instead asked ourselves: what if we just _let_ the agent reason about the work to be done and only provided the tools + guardrails to gather context and perform accounting work?
Turns out frontier models are GOOD at what they do, they outperformed our highly prescriptive skills and were able to work across a larger set of tasks even without instruction on how to do those tasks.
It\u2019s a breath of fresh air from the decade of CRUD I\u2019ve worked on, harness engineering is very neat.","title":null,"type":"comment","url":null},{"author":"xrd","children":[{"author":"gf000","children":[{"author":"xrd","children":[],"created_at":"2026-08-26T00:24:53.000Z","created_at_i":1787703893,"id":49442633,"options":[],"parent_id":49420163,"points":null,"story_id":49409092,"text":"I basically do this, tmux via tailscale. But, I often get inspiration and want to jump into a webapp from my phone, or review.
I've been playing with pi and the remote webui extension. I don't love it; it has a lot of chrome that obscures what I want to do. I just want a simple way to review the progress so far, and keep tweaking with minimal setup. Then, jump back into tmux when I'm back on my computer.
Thanks for your comments.","title":null,"type":"comment","url":null}],"created_at":"2026-08-24T14:18:23.000Z","created_at_i":1787581103,"id":49420163,"options":[],"parent_id":49410304,"points":null,"story_id":49409092,"text":"Not really solving all your cases, but I found tmux (or herdr) on a home server works pretty well. I just ssh into my home server and continue where I left off with the same claude/opencode open.
On a longer term, I think "assistant" style harnesses might help here, like vellum.ai. I no longer use that, but I asked it to create an ACP proxy through iroh (basically tailscale but on the application layer), and it managed to control claude on another device of mine. A friend did similar stuff with tailscale.
I have started writing a hobby harness with a web interface where I would like to support this "ACP proxy" mode natively, and also to make the models aware of different devices in some way and "move" work between them.","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T16:48:32.000Z","created_at_i":1787503712,"id":49410304,"options":[],"parent_id":49409092,"points":null,"story_id":49409092,"text":"Does anyone have a suggestion for a harness that is good at handoff?
When I say handoff, I mean:
* handoff from a terminal CLI to webui (on a phone)? \n * handoff from one team member, to another?\n * handoff from one communication modality, like writing a prompt in a TUI, to email? \n * handoff from one model to another, or one provider (openrouter)( to another (llama.cpp)\nI used to think that a PR would be a good place to centralize all this. Who cares what IDE, or developer, or location. But, now I feel like an agent harness might contain that better.
Why do I want handoff? I keep losing context of where my harness is running. Sometimes I am inside an isolated VM. Sometimes I'm on my laptop, sometimes I'm on my home machine with the big GPU for local models. If I could spin up a harness that could identify itself inside my tailscale network, then I could probably have a single web UI which allows me to keep all that context straight.
I'm tempted to experiment with Pi to configure such a thing. But, perhaps there are patterns out there already with a harness I have not considered.","title":null,"type":"comment","url":null},{"author":"kmansm27","children":[],"created_at":"2026-08-23T18:27:16.000Z","created_at_i":1787509636,"id":49411174,"options":[],"parent_id":49409092,"points":null,"story_id":49409092,"text":"From these comments, it seems like people still don't understand what harnesses are... The point is you shouldn't build a harness, you should use a harness and change its system prompt, the tools it has, MCPs it has, give it skills, etc, to make it work for your usecase. You aren't "building a harness on top of pi" if all you're doing is the above. You're just using the harness to connect different things to the LLM.","title":null,"type":"comment","url":null},{"author":"ni10c","children":[{"author":"asQuirreL","children":[{"author":"johnmw","children":[{"author":"mijowi","children":[],"created_at":"2026-08-25T06:48:49.000Z","created_at_i":1787640529,"id":49429949,"options":[],"parent_id":49414083,"points":null,"story_id":49409092,"text":"Although obviously unserious, I would argue a baby harness is mostly the opposite of what an agent harness does. A baby harness constrains and contains the baby, while an agent harness controls but also empowers the agent.","title":null,"type":"comment","url":null}],"created_at":"2026-08-24T00:49:47.000Z","created_at_i":1787532587,"id":49414083,"options":[],"parent_id":49411701,"points":null,"story_id":49409092,"text":"I wonder if I'm the only person whose first analogy that came to mind was: model = toddler, harness = baby bouncing jumper harness","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T19:28:09.000Z","created_at_i":1787513289,"id":49411701,"options":[],"parent_id":49411339,"points":null,"story_id":49409092,"text":"The first analogy that comes to mind, growing out of "harness", is more like harness = harness, model = horse (rather than harness as in climbing harness).
I guess you could say that tokens = hay, and agent = horse and cart, from there? Not sure how useful the hay part is but you could observe from the second that there are many different things you could harness a horse to (also a plough, or a coach, or just a saddle) based on your goal.","title":null,"type":"comment","url":null},{"author":"troyvit","children":[{"author":"ni10c","children":[],"created_at":"2026-08-24T16:16:39.000Z","created_at_i":1787588199,"id":49421975,"options":[],"parent_id":49411715,"points":null,"story_id":49409092,"text":"Thanks for these thoughts. Super insightful and appreciated","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T19:29:27.000Z","created_at_i":1787513367,"id":49411715,"options":[],"parent_id":49411339,"points":null,"story_id":49409092,"text":"I'm a climber so I'm biased but I really liked your climbing harness example because of the configuration you're able to easily make to the harness.
Saying the harness is like a car's chassis doesn't work as well for me because the chassis isn't as configurable as a climbing harness for as little work.
Getting deeper into the climbing analogy you can even swap out the harnesses themselves for wildly different climbs. Like using Claude Code with a bunch of agents for medical software (climbing K2 where that extra padding comes in super handy) and pi.dev with a local model for a respectable web project (sport route where you'll be back in a few hours and it's safe to be a little more exposed).
I'm glad your article made HN, and thank you for pi!","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T18:48:06.000Z","created_at_i":1787510886,"id":49411339,"options":[],"parent_id":49409092,"points":null,"story_id":49409092,"text":"Author here. It\u2019s ironic because this post was clearly geared towards non-hackers. But now that we\u2019re here.. the other analogy I considered presenting was:
harness = chassis,\nmodel = engine,\nfuel = tokens,\nagent = car
I\u2019m curious what y\u2019all might think and whether that analogy carries more explanatory power","title":null,"type":"comment","url":null},{"author":"sysfiend","children":[],"created_at":"2026-08-24T21:20:24.000Z","created_at_i":1787606424,"id":49425993,"options":[],"parent_id":49409092,"points":null,"story_id":49409092,"text":"Model -> brain cells' conections\nHarness -> everything else
I've been working with different setups in parallel for months (openclaw, pi, cursor per project harnesses and codex) and, even though using the same model most of the time, I can clearly see how the behave in very different ways deppending on the setup.
As a language model, language is our way to communicate and build everything around the models, which makes my younger self (who loved writing stories) very very happy :)","title":null,"type":"comment","url":null}],"created_at":"2026-08-23T14:24:21.000Z","created_at_i":1787495061,"id":49409092,"options":[],"parent_id":null,"points":589,"story_id":49409092,"text":null,"title":"What Is a Harness?","type":"story","url":"https://earendil.com/posts/what-is-a-harness/"}