Vequil is an observability layer that ingests structured action logs from runtimes like OpenClaw, Claude, LangChain, and OpenAI. We apply deterministic rules to catch missing auth keys, runaway loops, duplicate execution, and cost spikes. If an anomaly triggers, an AI diagnosis engine explains the failure in plain English and recommends operator actions.
It is runtime-agnostic\u2014any system that can POST JSON can send activity to our ingest API. I'd appreciate feedback from anyone here running agents in production on our anomaly detection rules and ingestion pipeline."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"[dead]"}},"_tags":["comment","author_nxd9144","story_47704834"],"author":"nxd9144","comment_text":"Shipping AI agents right now means deploying autonomous code with no accountability layer beneath it. I\u2019m building Vequil to solve this.
Vequil is an observability layer that ingests structured action logs from runtimes like OpenClaw, Claude, LangChain, and OpenAI. We apply deterministic rules to catch missing auth keys, runaway loops, duplicate execution, and cost spikes. If an anomaly triggers, an AI diagnosis engine explains the failure in plain English and recommends operator actions.
It is runtime-agnostic\u2014any system that can POST JSON can send activity to our ingest API. I'd appreciate feedback from anyone here running agents in production on our anomaly detection rules and ingestion pipeline.","created_at":"2026-04-09T15:14:22Z","created_at_i":1775747662,"objectID":"47704835","parent_id":47704834,"story_id":47704834,"story_title":"[dead]","updated_at":"2026-04-09T15:15:47Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ylohnitram"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"Hey HN,
I am building Agentis, an AI-native programming language written in vanilla Rust. Instead of treating prompts as external library calls, "everything is a prompt" is a core language primitive.
There is no standard library. No string.split(). If an agent needs to process data, it asks the LLM.
I also completely fused the language with a VCS. Code isn't stored in text files; it's a binary, SHA-256-hashed DAG.
A few core concepts I implemented:
1. Cognitive Budget: Every operation costs fuel. This prevents runaway agent loops and forces efficient prompt design.
2. Evolutionary Branching: I use explore blocks to fork execution. If a solution passes the validate block, the branch survives. If it fails, it's silently discarded.
3. Content-Addressed Code: Since the AST is hashed, there are no merge conflicts. You import by hash.
I just completed Phase 4 of my implementation and am currently gearing up for Phase 5. The CLI supports local models (Ollama), Claude, Anthropic API, etc.
It\u2019s highly experimental, and I'm at the stage where I'd love to discuss these core concepts with the community and find people interested in collaborating.
I'd love to hear your thoughts on this architecture, the evolutionary branching, or the VCS model."},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["ai"],"value":"Agentis \u2013 An AI-native programming language where the LLM is the stdlib"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/Replikanti/agentis"}},"_tags":["comment","author_ylohnitram","story_47317010"],"author":"ylohnitram","comment_text":"Hey HN,
I am building Agentis, an AI-native programming language written in vanilla Rust. Instead of treating prompts as external library calls, "everything is a prompt" is a core language primitive.
There is no standard library. No string.split(). If an agent needs to process data, it asks the LLM.
I also completely fused the language with a VCS. Code isn't stored in text files; it's a binary, SHA-256-hashed DAG.
A few core concepts I implemented:
1. Cognitive Budget: Every operation costs fuel. This prevents runaway agent loops and forces efficient prompt design.
2. Evolutionary Branching: I use explore blocks to fork execution. If a solution passes the validate block, the branch survives. If it fails, it's silently discarded.
3. Content-Addressed Code: Since the AST is hashed, there are no merge conflicts. You import by hash.
I just completed Phase 4 of my implementation and am currently gearing up for Phase 5. The CLI supports local models (Ollama), Claude, Anthropic API, etc.
It\u2019s highly experimental, and I'm at the stage where I'd love to discuss these core concepts with the community and find people interested in collaborating.
I'd love to hear your thoughts on this architecture, the evolutionary branching, or the VCS model.","created_at":"2026-03-09T23:07:42Z","created_at_i":1773097662,"objectID":"47317011","parent_id":47317010,"story_id":47317010,"story_title":"Agentis \u2013 An AI-native programming language where the LLM is the stdlib","story_url":"https://github.com/Replikanti/agentis","updated_at":"2026-03-09T23:11:57Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nordic_lion"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"If you are running AI workloads/agents or LLM-backed systems in production, how do you actually shut one down when it starts behaving badly?
By \u201cmisbehaving\u201d I mean things like:\n-runaway spend\n-latency issues\n-prompt loops\n-tool abuse or unexpected external calls\n-data leakage risks\n-cascading failures across downstream services
In most systems I\u2019ve seen, there is good observability. You can see logs, traces, cost dashboards. But the actual shutdown mechanism often ends up being manual: disable a feature flag, revoke an API key, roll back a deployment, rate limit something upstream.
I am trying to understand what people are doing in practice.
-What is your actual kill mechanism?\n-Is it bound to a model endpoint, an agent instance, a workflow, a Kubernetes workload, something else?\n-Is shutdown automated under certain conditions, or always human-approved?\n-What did you discover only after your first real incident?
Concrete examples would be extremely helpful."},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["ai"],"value":"Ask HN: How do you shut down misbehaving AI in production?"}},"_tags":["story","author_nordic_lion","story_47002748","ask_hn"],"author":"nordic_lion","children":[47002813,47003491],"created_at":"2026-02-13T13:57:04Z","created_at_i":1770991024,"num_comments":7,"objectID":"47002748","points":2,"story_id":47002748,"story_text":"If you are running AI workloads/agents or LLM-backed systems in production, how do you actually shut one down when it starts behaving badly?
By \u201cmisbehaving\u201d I mean things like:\n-runaway spend\n-latency issues\n-prompt loops\n-tool abuse or unexpected external calls\n-data leakage risks\n-cascading failures across downstream services
In most systems I\u2019ve seen, there is good observability. You can see logs, traces, cost dashboards. But the actual shutdown mechanism often ends up being manual: disable a feature flag, revoke an API key, roll back a deployment, rate limit something upstream.
I am trying to understand what people are doing in practice.
-What is your actual kill mechanism?\n-Is it bound to a model endpoint, an agent instance, a workflow, a Kubernetes workload, something else?\n-Is shutdown automated under certain conditions, or always human-approved?\n-What did you discover only after your first real incident?
Concrete examples would be extremely helpful.","title":"Ask HN: How do you shut down misbehaving AI in production?","updated_at":"2026-03-05T23:31:42Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"lexokoh"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"Hi all, I'm Dante, and I'm building Echos, a platform that gives you pre-built AI agents so you can stop rebuilding orchestrators, database agents, and retry logic every time.
What it does:
- Pre-built agents: Database queries, API calls, web search, data analysis, code generation.
- YAML-based workflows: Define your agent architecture without rebuilding orchestrators.
- Built-in guardrails: SQL injection protection, SSRF blocking, table/domain whitelisting.
- Visual traces: See what happened, where it failed, and how much it cost.
Why I built it:
Every time I build a multi-agent system, I spend 2-3 weeks creating the same infrastructure: orchestrators that route tasks, database agents with SQL guardrails, retry logic, loop limiting, and cost tracking. Then another week of debugging when things break. I wanted to ship features, not plumbing.
Most frameworks are bulky and complex. You just want pre-built components you can compose like AWS services.
What Echos gives you:
- Ship faster: Pre-built agents you compose in YAML.
- Debug in minutes: Visual traces show exactly what happened, where it failed, and how much it cost.
- Prevent disasters: Built-in guardrails (SQL injection protection, SSRF blocking, loop limiting) catch 80% of dangerous operations.
- Control costs: Per-agent spending limits prevent runaway bills.
Try it:\nClone https://github.com/treadiehq/echos or go to https://echoshq.com
import { EchosRuntime } from '@echoshq/runtime';
const runtime = new EchosRuntime({\n apiKey: process.env.ECHOS_API_KEY,\n apiUrl: process.env.ECHOS_API_URL,\n workflow: './workflow.yaml' // Define agents and routes in YAML\n});
// Simple usage\nawait runtime.run({\n task: 'Analyze customer churn',\n memory: { year: 2024, region: 'north' }\n});
Tech:
- NestJS for the backend API: Needed structured DI and middleware for auth.
- Postgres for trace storage: JSON columns for flexible span logs, native SQL performance.
- Resend for magic link authentication: Reliable email delivery without managing SMTP.
- Nuxt 3 for the dashboard: SSR for fast initial load, client-side interactivity for live traces.
- Railway for deployment: Fast deploys. First time trying it. My previous default is Digital Ocean.
What I learned:
- Time saved is the real value: Teams don't want another framework, they want to ship faster.
- Debugging is 50% of the work: Visual traces that show the full execution path are essential.
- Simple guardrails work: Blocking DELETE/DROP and unknown domains catches most disasters.
- YAML > Code for config: Non-engineers can edit workflows without touching code.
Looking for feedback:
- Does this solve a real problem for you?
- Which agents would you use most? database, API calls, web search, data analysis, or code generation?
- Is YAML configuration expressive enough, or do you need more programmatic control?
- What agents should we add next? (GitHub, Slack, email, cloud APIs?)
- Would you use this for autonomous agents, or just one-off tasks?
- Would this save you time on your next multi-agent project?
- What's missing that would make this immediately useful?
Thank you!"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["ai","agent"],"value":"Show HN: Echos \u2013 A lightweight multi-agent AI system with pre-built agents"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/treadiehq/echos"}},"_tags":["story","author_lexokoh","story_45883228","show_hn"],"author":"lexokoh","created_at":"2025-11-11T01:52:52Z","created_at_i":1762825972,"num_comments":0,"objectID":"45883228","points":2,"story_id":45883228,"story_text":"Hi all, I'm Dante, and I'm building Echos, a platform that gives you pre-built AI agents so you can stop rebuilding orchestrators, database agents, and retry logic every time.
What it does:
- Pre-built agents: Database queries, API calls, web search, data analysis, code generation.
- YAML-based workflows: Define your agent architecture without rebuilding orchestrators.
- Built-in guardrails: SQL injection protection, SSRF blocking, table/domain whitelisting.
- Visual traces: See what happened, where it failed, and how much it cost.
Why I built it:
Every time I build a multi-agent system, I spend 2-3 weeks creating the same infrastructure: orchestrators that route tasks, database agents with SQL guardrails, retry logic, loop limiting, and cost tracking. Then another week of debugging when things break. I wanted to ship features, not plumbing.
Most frameworks are bulky and complex. You just want pre-built components you can compose like AWS services.
What Echos gives you:
- Ship faster: Pre-built agents you compose in YAML.
- Debug in minutes: Visual traces show exactly what happened, where it failed, and how much it cost.
- Prevent disasters: Built-in guardrails (SQL injection protection, SSRF blocking, loop limiting) catch 80% of dangerous operations.
- Control costs: Per-agent spending limits prevent runaway bills.
Try it:\nClone https://github.com/treadiehq/echos or go to https://echoshq.com
import { EchosRuntime } from '@echoshq/runtime';
const runtime = new EchosRuntime({\n apiKey: process.env.ECHOS_API_KEY,\n apiUrl: process.env.ECHOS_API_URL,\n workflow: './workflow.yaml' // Define agents and routes in YAML\n});
// Simple usage\nawait runtime.run({\n task: 'Analyze customer churn',\n memory: { year: 2024, region: 'north' }\n});
Tech:
- NestJS for the backend API: Needed structured DI and middleware for auth.
- Postgres for trace storage: JSON columns for flexible span logs, native SQL performance.
- Resend for magic link authentication: Reliable email delivery without managing SMTP.
- Nuxt 3 for the dashboard: SSR for fast initial load, client-side interactivity for live traces.
- Railway for deployment: Fast deploys. First time trying it. My previous default is Digital Ocean.
What I learned:
- Time saved is the real value: Teams don't want another framework, they want to ship faster.
- Debugging is 50% of the work: Visual traces that show the full execution path are essential.
- Simple guardrails work: Blocking DELETE/DROP and unknown domains catches most disasters.
- YAML > Code for config: Non-engineers can edit workflows without touching code.
Looking for feedback:
- Does this solve a real problem for you?
- Which agents would you use most? database, API calls, web search, data analysis, or code generation?
- Is YAML configuration expressive enough, or do you need more programmatic control?
- What agents should we add next? (GitHub, Slack, email, cloud APIs?)
- Would you use this for autonomous agents, or just one-off tasks?
- Would this save you time on your next multi-agent project?
- What's missing that would make this immediately useful?
Thank you!","title":"Show HN: Echos \u2013 A lightweight multi-agent AI system with pre-built agents","updated_at":"2026-03-29T18:08:02Z","url":"https://github.com/treadiehq/echos"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dipampaul17"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"Your AI agent hits an infinite loop and racks up $2000 in API charges overnight. This happens weekly to AI developers.
AgentGuard monitors API calls in real-time and automatically kills your process when it hits your budget limit.
How it works:
Add 2 lines to any AI project:
const agentGuard = require('agent-guard');\n await agentGuard.init({ limit: 50 }); // $50 budget\n\n // Your existing code runs unchanged\n const response = await openai.chat.completions.create({...});\n // AgentGuard tracks costs automatically\nWhy I built this:
I got tired of seeing "I accidentally spent $500 on OpenAI" posts. Existing tools like tokencost help you measure costs after the fact, but nothing prevents runaway spending in real-time.
AgentGuard is essentially a circuit breaker for AI API costs. It's saved me from several costly bugs during development.
Limitations: Only works with OpenAI and Anthropic APIs currently. Cost calculations are estimates based on documented pricing.
Source: https://github.com/dipampaul17/AgentGuard
Install: npm i agent-guard"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["ai","agent"],"value":"Show HN: AgentGuard \u2013 Auto-kill AI agents before they burn through your budget"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/dipampaul17/AgentGuard"}},"_tags":["story","author_dipampaul17","story_44742710","show_hn"],"author":"dipampaul17","children":[44743048,44743049,44743095,44743606,44743689,44743962,44744000,44745758,44746104],"created_at":"2025-07-31T05:54:04Z","created_at_i":1753941244,"num_comments":26,"objectID":"44742710","points":47,"story_id":44742710,"story_text":"Your AI agent hits an infinite loop and racks up $2000 in API charges overnight. This happens weekly to AI developers.
AgentGuard monitors API calls in real-time and automatically kills your process when it hits your budget limit.
How it works:
Add 2 lines to any AI project:
const agentGuard = require('agent-guard');\n await agentGuard.init({ limit: 50 }); // $50 budget\n\n // Your existing code runs unchanged\n const response = await openai.chat.completions.create({...});\n // AgentGuard tracks costs automatically\nWhy I built this:
I got tired of seeing "I accidentally spent $500 on OpenAI" posts. Existing tools like tokencost help you measure costs after the fact, but nothing prevents runaway spending in real-time.
AgentGuard is essentially a circuit breaker for AI API costs. It's saved me from several costly bugs during development.
Limitations: Only works with OpenAI and Anthropic APIs currently. Cost calculations are estimates based on documented pricing.
Source: https://github.com/dipampaul17/AgentGuard
Install: npm i agent-guard","title":"Show HN: AgentGuard \u2013 Auto-kill AI agents before they burn through your budget","updated_at":"2026-05-13T05:08:48Z","url":"https://github.com/dipampaul17/AgentGuard"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"PrateekRao01"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"Hi HN \u2014 I\u2019m Prateek Rao. My cofounders and I built Cortexa, which we describe as a Bloomberg terminal for agentic memory.
A pattern I keep seeing: when agents misbehave, most teams iterate on prompts and then \u201cfix\u201d it by plugging in a memory layer (vector DB + RAG). That helps sometimes \u2014 but it doesn\u2019t guarantee correctness. In practice it often introduces a new failure mode: the agent retrieves something dubious, writes it back to memory as if it\u2019s truth, and that mistake becomes sticky. Over time you get memory pollution, circular hallucination loops, and debugging turns into log archaeology.
What Cortexa does:
1. Agent decision forensics (end-to-end \u201cwhy\u201d): trace outputs/actions back to the exact retrievals, memory writes, and tool calls that caused them.
2. Memory write governance: intercept and score memory writes (0\u20131), and optionally block/quarantine ungrounded entries before they poison future runs.
3. Memory hygiene + vector store noise control: automatically detect and remove near-duplicate / low-signal entries so retrieval stays high-quality and storage + inference costs don\u2019t creep up.
Why this matters:\nObservability is the missing layer for agentic AI. Without it, autonomy is fragile: small errors silently compound, deployments become risky, and engineering cost goes up because failures aren\u2019t reproducible or attributable.
Who this is for:\n1. Teams shipping agentic workflows in production\n2. Anyone fighting \u201cunknown why\u201d failures, memory pollution, or runaway context costs\n3. Engineers who want auditability + faster debugging loops
Site: https://cortexa.ink/
Would love feedback from anyone running agents at scale:\n1.What\u2019s the most painful agent failure mode you\u2019ve seen in production?\n2.What signals would you want in an \u201cagent terminal\u201d (retrieval diffs, memory blame, tool-call traces, alerts, etc.)?"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Cortexa \u2013 Bloomberg terminal for agentic memory"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://cortexa.ink"}},"_tags":["story","author_PrateekRao01","story_47228173","show_hn"],"author":"PrateekRao01","children":[47228186,47236075,47308078,47406432],"created_at":"2026-03-03T04:45:10Z","created_at_i":1772513110,"num_comments":4,"objectID":"47228173","points":10,"story_id":47228173,"story_text":"Hi HN \u2014 I\u2019m Prateek Rao. My cofounders and I built Cortexa, which we describe as a Bloomberg terminal for agentic memory.
A pattern I keep seeing: when agents misbehave, most teams iterate on prompts and then \u201cfix\u201d it by plugging in a memory layer (vector DB + RAG). That helps sometimes \u2014 but it doesn\u2019t guarantee correctness. In practice it often introduces a new failure mode: the agent retrieves something dubious, writes it back to memory as if it\u2019s truth, and that mistake becomes sticky. Over time you get memory pollution, circular hallucination loops, and debugging turns into log archaeology.
What Cortexa does:
1. Agent decision forensics (end-to-end \u201cwhy\u201d): trace outputs/actions back to the exact retrievals, memory writes, and tool calls that caused them.
2. Memory write governance: intercept and score memory writes (0\u20131), and optionally block/quarantine ungrounded entries before they poison future runs.
3. Memory hygiene + vector store noise control: automatically detect and remove near-duplicate / low-signal entries so retrieval stays high-quality and storage + inference costs don\u2019t creep up.
Why this matters:\nObservability is the missing layer for agentic AI. Without it, autonomy is fragile: small errors silently compound, deployments become risky, and engineering cost goes up because failures aren\u2019t reproducible or attributable.
Who this is for:\n1. Teams shipping agentic workflows in production\n2. Anyone fighting \u201cunknown why\u201d failures, memory pollution, or runaway context costs\n3. Engineers who want auditability + faster debugging loops
Site: https://cortexa.ink/
Would love feedback from anyone running agents at scale:\n1.What\u2019s the most painful agent failure mode you\u2019ve seen in production?\n2.What signals would you want in an \u201cagent terminal\u201d (retrieval diffs, memory blame, tool-call traces, alerts, etc.)?","title":"Show HN: Cortexa \u2013 Bloomberg terminal for agentic memory","updated_at":"2026-03-16T23:23:39Z","url":"https://cortexa.ink"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Zephyr0x"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"I just learned a $37,901.73 lesson about AWS Bedrock, Claude Opus, prompt caching, and the complete lack of hard safety rails around metered AI infrastructure.
This was not a leaked key.\nThis was not crypto mining.\nThis was not an infinite loop.\nThis was not one ridiculous request.
It was a normal local coding-agent workflow:
Droid -> OpenAI-compatible API -> LiteLLM -> AWS Bedrock -> Claude Opus 4.6
I assumed prompt caching was working because every layer in the chain made that assumption feel reasonable:
- Claude supports prompt caching\n- Bedrock supports prompt caching for Claude\n- LiteLLM supports Bedrock\n- Droid can talk to an OpenAI-compatible endpoint
But the bill told a different story.
The gross Opus usage was $37,901.73. AWS credits covered about $8,026.54, leaving roughly $29,875.19 net.
The expensive line item was not output. It was repeated uncached input:
- uncached input tokens: ~6.47B tokens, ~$35.6k\n- cache read input tokens: ~1.67B tokens, ~$918\n- cache write input tokens: ~101M tokens, ~$698\n- output tokens: ~25M tokens, ~$698
So yes, there was some cache activity. But nowhere near enough for a high-frequency agent workflow.
The thing that makes me angry is that all of this was allowed to fail silently.
\u201cPrompt caching is supported\u201d is not the same as \u201cyour actual agent stack is using prompt caching correctly.\u201d
\u201cBudget alerts are configured\u201d is not the same as \u201cspend will stop.\u201d
\u201cCredits are applied\u201d is not the same as \u201cyou will notice the bad cost structure early.\u201d
These are soft signals pretending to be safety boundaries.
For LLM agents, that is not good enough.
A coding agent can run while you sleep. It can carry huge context. It can repeatedly send repo state, tool schemas, instructions, history, and file contents. If caching is misconfigured or partially effective, the failure mode is not a small inefficiency. It is a runaway cloud bill.
Why is there no simple hard cap at the Bedrock/API level?
Why can I not say:
- this IAM principal may spend at most $200/month\n- this model may not be called more than N times/day\n- this workflow may not send more than N uncached input tokens/hour\n- stop serving requests once the budget is crossed
Cloud providers have had decades to learn that \u201cemail me after the money is gone\u201d is not a safety mechanism.
I am not saying AWS, Anthropic, LiteLLM, or Droid alone caused this. I made the mistake of putting a metered premium model into a daily agent workflow before putting hard guardrails around it.
But that is exactly the point: the platform lets a very normal integration mistake turn into a car-sized invoice.
My takeaway:
Prompt caching is not a checkbox.\nBudget alerts are not a kill switch.\nCredits are not protection.\nAnd metered AI backends need hard spend limits before agents become normal infrastructure.
Has anyone here built reliable guardrails for this? IAM deny rules? API gateways? token-budget proxies? per-workflow kill switches?
Because right now the default feels absurdly dangerous."},"title":{"matchLevel":"none","matchedWords":[],"value":"$38k AWS Bedrock bill caused by a simple prompt caching miss"}},"_tags":["story","author_Zephyr0x","story_47933355","ask_hn"],"author":"Zephyr0x","children":[47961996,47983956,47997931],"created_at":"2026-04-28T12:07:40Z","created_at_i":1777378060,"num_comments":0,"objectID":"47933355","points":8,"story_id":47933355,"story_text":"I just learned a $37,901.73 lesson about AWS Bedrock, Claude Opus, prompt caching, and the complete lack of hard safety rails around metered AI infrastructure.
This was not a leaked key.\nThis was not crypto mining.\nThis was not an infinite loop.\nThis was not one ridiculous request.
It was a normal local coding-agent workflow:
Droid -> OpenAI-compatible API -> LiteLLM -> AWS Bedrock -> Claude Opus 4.6
I assumed prompt caching was working because every layer in the chain made that assumption feel reasonable:
- Claude supports prompt caching\n- Bedrock supports prompt caching for Claude\n- LiteLLM supports Bedrock\n- Droid can talk to an OpenAI-compatible endpoint
But the bill told a different story.
The gross Opus usage was $37,901.73. AWS credits covered about $8,026.54, leaving roughly $29,875.19 net.
The expensive line item was not output. It was repeated uncached input:
- uncached input tokens: ~6.47B tokens, ~$35.6k\n- cache read input tokens: ~1.67B tokens, ~$918\n- cache write input tokens: ~101M tokens, ~$698\n- output tokens: ~25M tokens, ~$698
So yes, there was some cache activity. But nowhere near enough for a high-frequency agent workflow.
The thing that makes me angry is that all of this was allowed to fail silently.
\u201cPrompt caching is supported\u201d is not the same as \u201cyour actual agent stack is using prompt caching correctly.\u201d
\u201cBudget alerts are configured\u201d is not the same as \u201cspend will stop.\u201d
\u201cCredits are applied\u201d is not the same as \u201cyou will notice the bad cost structure early.\u201d
These are soft signals pretending to be safety boundaries.
For LLM agents, that is not good enough.
A coding agent can run while you sleep. It can carry huge context. It can repeatedly send repo state, tool schemas, instructions, history, and file contents. If caching is misconfigured or partially effective, the failure mode is not a small inefficiency. It is a runaway cloud bill.
Why is there no simple hard cap at the Bedrock/API level?
Why can I not say:
- this IAM principal may spend at most $200/month\n- this model may not be called more than N times/day\n- this workflow may not send more than N uncached input tokens/hour\n- stop serving requests once the budget is crossed
Cloud providers have had decades to learn that \u201cemail me after the money is gone\u201d is not a safety mechanism.
I am not saying AWS, Anthropic, LiteLLM, or Droid alone caused this. I made the mistake of putting a metered premium model into a daily agent workflow before putting hard guardrails around it.
But that is exactly the point: the platform lets a very normal integration mistake turn into a car-sized invoice.
My takeaway:
Prompt caching is not a checkbox.\nBudget alerts are not a kill switch.\nCredits are not protection.\nAnd metered AI backends need hard spend limits before agents become normal infrastructure.
Has anyone here built reliable guardrails for this? IAM deny rules? API gateways? token-budget proxies? per-workflow kill switches?
Because right now the default feels absurdly dangerous.","title":"$38k AWS Bedrock bill caused by a simple prompt caching miss","updated_at":"2026-05-06T12:37:42Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"vira28"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"A couple of early-to-mid-stage startups I'm consulting with are asking the same question: their AI/ML team wants production Postgres data, and nobody's quite sure how to give it to them.
I've handled this before for BI teams \u2014 read replica with a generous `max_standby_streaming_delay` and `hot_standby_feedback` on, accepting the occasional bloat on the primary. Worked fine. But the AI/ML ask feels different in ways I can't fully articulate yet, which is part of why I'm asking.
A few things I'm trying to calibrate:
Where does the agent actually connect? Primary with RLS, read replica, warehouse (Snowflake/BigQuery/Redshift), lakehouse (Iceberg/Delta on S3), or something else?
If you're not doing this \u2014 is it compliance, cost fear, bad experiences (runaway queries, PII in prompts), or something else?
And the one I'm most curious about: does this actually feel different from giving BI tools DB access, or is it the same problem wearing new clothes?
Not looking for product recommendations. Trying to get a real sense from people who've actually faced this challenge."},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["ai","agent"],"value":"Ask HN: Is giving AI agents DB access the new BI-tool problem?"}},"_tags":["story","author_vira28","story_47827486","ask_hn"],"author":"vira28","children":[47827678,47829034,47831915,47831988,47887796,47887799],"created_at":"2026-04-19T20:44:39Z","created_at_i":1776631479,"num_comments":2,"objectID":"47827486","points":6,"story_id":47827486,"story_text":"A couple of early-to-mid-stage startups I'm consulting with are asking the same question: their AI/ML team wants production Postgres data, and nobody's quite sure how to give it to them.
I've handled this before for BI teams \u2014 read replica with a generous `max_standby_streaming_delay` and `hot_standby_feedback` on, accepting the occasional bloat on the primary. Worked fine. But the AI/ML ask feels different in ways I can't fully articulate yet, which is part of why I'm asking.
A few things I'm trying to calibrate:
Where does the agent actually connect? Primary with RLS, read replica, warehouse (Snowflake/BigQuery/Redshift), lakehouse (Iceberg/Delta on S3), or something else?
If you're not doing this \u2014 is it compliance, cost fear, bad experiences (runaway queries, PII in prompts), or something else?
And the one I'm most curious about: does this actually feel different from giving BI tools DB access, or is it the same problem wearing new clothes?
Not looking for product recommendations. Trying to get a real sense from people who've actually faced this challenge.","title":"Ask HN: Is giving AI agents DB access the new BI-tool problem?","updated_at":"2026-04-24T09:59:29Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"sahiljagtapyc"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"AgentBudget \u2014 I got a $32 surprise bill from a runaway AI agent, so I built a real-time kill switch
A few weeks ago I left an AI agent running, walked away for 20 minutes, and came back to a bill I didn't expect. I searched for a tool that could just stop the agent when it hit a dollar limit. Nothing existed.\nSo I built AgentBudget. It monkey-patches the OpenAI and Anthropic SDKs, tracks running cost per API call, and throws a BudgetExceeded exception the moment the agent crosses its limit. Under 2ms overhead. One import, one line.\nfrom agentbudget import BudgetManager\nbudget = BudgetManager(max_cost="$5.00")\nThat's it. Works with OpenAI, Anthropic, Gemini, Mistral, Cohere, LangChain, and CrewAI.\nWould love feedback from people building agents in production \u2014 especially around what the cloud dashboard should look like.
GitHub: github.com/sahiljagtap08/agentbudget"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"I kept getting surprise API bills from my agents"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://agentbudget.dev"}},"_tags":["comment","author_sahiljagtapyc","story_47418574"],"author":"sahiljagtapyc","comment_text":"AgentBudget \u2014 I got a $32 surprise bill from a runaway AI agent, so I built a real-time kill switch
A few weeks ago I left an AI agent running, walked away for 20 minutes, and came back to a bill I didn't expect. I searched for a tool that could just stop the agent when it hit a dollar limit. Nothing existed.\nSo I built AgentBudget. It monkey-patches the OpenAI and Anthropic SDKs, tracks running cost per API call, and throws a BudgetExceeded exception the moment the agent crosses its limit. Under 2ms overhead. One import, one line.\nfrom agentbudget import BudgetManager\nbudget = BudgetManager(max_cost="$5.00")\nThat's it. Works with OpenAI, Anthropic, Gemini, Mistral, Cohere, LangChain, and CrewAI.\nWould love feedback from people building agents in production \u2014 especially around what the cloud dashboard should look like.
GitHub: github.com/sahiljagtap08/agentbudget","created_at":"2026-03-17T21:28:32Z","created_at_i":1773782912,"objectID":"47418575","parent_id":47418574,"story_id":47418574,"story_title":"I kept getting surprise API bills from my agents","story_url":"https://agentbudget.dev","updated_at":"2026-06-15T01:15:42Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"rizzo94"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"You nailed it with the 'hype smell.' The silence in your circles is likely because the churn rate on OpenClaw is massive. Most people hit that 'Day 2' wall\u2014where the novelty wears off and the reality of securing a bot with shell access sets in\u2014and they just quietly shut it down.
I was in that exact boat (wanted the agency, didn't want the sysadmin headache). I\u2019ve actually pivoted to testing PAIO (Personal AI Operator) instead. It targets the same 'agentic' utility but uses a BYOK architecture and a managed security layer.
It basically solves the specific failures you linked:
Security: You aren't leaving a shell open on your local machine.
Setup: It\u2019s a one-click integration rather than a failed sandbox install.
Cost: BYOK means you control the token burn directly, so no surprise bills from a runaway loop.
It feels like the 'adult in the room' version of these experiments. Less dramatic stories, perhaps, but it actually runs daily without me worrying it\u2019s going to rm -rf my home directory."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Any real OpenClaw (Clawd Bot/Molt Bot) users? What's your experience?"}},"_tags":["comment","author_rizzo94","story_46838946"],"author":"rizzo94","comment_text":"You nailed it with the 'hype smell.' The silence in your circles is likely because the churn rate on OpenClaw is massive. Most people hit that 'Day 2' wall\u2014where the novelty wears off and the reality of securing a bot with shell access sets in\u2014and they just quietly shut it down.
I was in that exact boat (wanted the agency, didn't want the sysadmin headache). I\u2019ve actually pivoted to testing PAIO (Personal AI Operator) instead. It targets the same 'agentic' utility but uses a BYOK architecture and a managed security layer.
It basically solves the specific failures you linked:
Security: You aren't leaving a shell open on your local machine.
Setup: It\u2019s a one-click integration rather than a failed sandbox install.
Cost: BYOK means you control the token burn directly, so no surprise bills from a runaway loop.
It feels like the 'adult in the room' version of these experiments. Less dramatic stories, perhaps, but it actually runs daily without me worrying it\u2019s going to rm -rf my home directory.","created_at":"2026-02-03T06:52:54Z","created_at_i":1770101574,"objectID":"46867443","parent_id":46838946,"story_id":46838946,"story_title":"Ask HN: Any real OpenClaw (Clawd Bot/Molt Bot) users? What's your experience?","updated_at":"2026-03-05T23:29:59Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"willempienaar"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"Cleric | https://cleric.ai | Staff SWE, AI | Staff SWE, Product | Founding Marketer | Staff Designer | SWE, Backend | SF Onsite (Designer Remote)
Engineers can't keep up with their production environments: dozens of services, dashboards everywhere, alerts firing constantly. The information to diagnose most issues already exists, but finding it takes longer than fixing the problem. And the more AI-generated code ships, the more services get deployed by people who won't be around to debug them.
Cleric connects to your existing observability stack, autonomously investigates production incidents, and tells engineers what's wrong. We're well funded with years of runway, a small team of AI and infrastructure veterans in SF, growing quickly. Stack: Python, Go, LLMs, Kubernetes.
Some of the problems we work on:
There's no test suite in production. When the AI says "the root cause is X," how do you verify that? You can't A/B test diagnoses. Ground truth labels don't exist. We build evaluation systems that track resolution outcomes over weeks and correlate fixes with diagnoses to build statistical confidence.
When something breaks, everything looks broken. Database latency spikes, five services throw errors, CPU goes up, logs explode. When an agent sees 47 anomalies at once, it needs to figure out which one is the root cause and which are symptoms, across systems with feedback loops, hidden dependencies, and non-obvious temporal relationships.
A single investigation might need six hours of metrics across 50 services, 10GB of logs, 10,000 distributed traces, the last 30 deployments, and the relevant runbooks. LLMs have finite context windows. What's relevant isn't known until you investigate. Getting retrieval wrong means wrong conclusions or exploding costs.
We're hiring:
- Staff Software Engineer, AI: You build the core agent. Reasoning, evals, self-improving feedback loops. You debug agent behavior by tracing reasoning and tool choices to understand why the agent made a specific decision. You build the systems that make a non-deterministic agent reliable, and push it to handle increasingly complex incidents.
- Staff Software Engineer, Product: You define what an AI SRE should actually be. When AI handles the reasoning, how do engineers stay sharp for cases it can't handle? How do you build trust when someone needs to verify agent conclusions at 2AM? You answer these by embedding with customers during real incidents, running experiments, and making the technical calls to ship what works.
- Software Engineer, Backend: You build and scale the investigation platform alongside our senior engineers. Integrations with Datadog, PagerDuty, and dozens of observability tools. Agent reasoning pipelines. Runtime systems that handle real-time data streaming at scale. You'll ship customer-facing functionality across the stack.
- Founding Marketer: Software engineers are allergic to AI hype. Our users already love the product. The challenge is reaching the next thousand teams without setting off their BS detectors. You build the marketing function from scratch: programs, pipeline, infrastructure. You need a technical foundation and the ability to hold a 30-minute conversation with a platform engineering lead without getting lost.
- Staff Designer (Remote): The interfaces for AI agents don't exist yet. How do you make autonomous reasoning legible without overwhelming? You own design across brand, product, and marketing, defining the visual language for a new category.
Apply: https://jobs.ashbyhq.com/cleric | Email: willem-hn@cleric.ai
Willem, Co-founder & CTO"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Who is hiring? (February 2026)"}},"_tags":["comment","author_willempienaar","story_46857488"],"author":"willempienaar","comment_text":"Cleric | https://cleric.ai | Staff SWE, AI | Staff SWE, Product | Founding Marketer | Staff Designer | SWE, Backend | SF Onsite (Designer Remote)
Engineers can't keep up with their production environments: dozens of services, dashboards everywhere, alerts firing constantly. The information to diagnose most issues already exists, but finding it takes longer than fixing the problem. And the more AI-generated code ships, the more services get deployed by people who won't be around to debug them.
Cleric connects to your existing observability stack, autonomously investigates production incidents, and tells engineers what's wrong. We're well funded with years of runway, a small team of AI and infrastructure veterans in SF, growing quickly. Stack: Python, Go, LLMs, Kubernetes.
Some of the problems we work on:
There's no test suite in production. When the AI says "the root cause is X," how do you verify that? You can't A/B test diagnoses. Ground truth labels don't exist. We build evaluation systems that track resolution outcomes over weeks and correlate fixes with diagnoses to build statistical confidence.
When something breaks, everything looks broken. Database latency spikes, five services throw errors, CPU goes up, logs explode. When an agent sees 47 anomalies at once, it needs to figure out which one is the root cause and which are symptoms, across systems with feedback loops, hidden dependencies, and non-obvious temporal relationships.
A single investigation might need six hours of metrics across 50 services, 10GB of logs, 10,000 distributed traces, the last 30 deployments, and the relevant runbooks. LLMs have finite context windows. What's relevant isn't known until you investigate. Getting retrieval wrong means wrong conclusions or exploding costs.
We're hiring:
- Staff Software Engineer, AI: You build the core agent. Reasoning, evals, self-improving feedback loops. You debug agent behavior by tracing reasoning and tool choices to understand why the agent made a specific decision. You build the systems that make a non-deterministic agent reliable, and push it to handle increasingly complex incidents.
- Staff Software Engineer, Product: You define what an AI SRE should actually be. When AI handles the reasoning, how do engineers stay sharp for cases it can't handle? How do you build trust when someone needs to verify agent conclusions at 2AM? You answer these by embedding with customers during real incidents, running experiments, and making the technical calls to ship what works.
- Software Engineer, Backend: You build and scale the investigation platform alongside our senior engineers. Integrations with Datadog, PagerDuty, and dozens of observability tools. Agent reasoning pipelines. Runtime systems that handle real-time data streaming at scale. You'll ship customer-facing functionality across the stack.
- Founding Marketer: Software engineers are allergic to AI hype. Our users already love the product. The challenge is reaching the next thousand teams without setting off their BS detectors. You build the marketing function from scratch: programs, pipeline, infrastructure. You need a technical foundation and the ability to hold a 30-minute conversation with a platform engineering lead without getting lost.
- Staff Designer (Remote): The interfaces for AI agents don't exist yet. How do you make autonomous reasoning legible without overwhelming? You own design across brand, product, and marketing, defining the visual language for a new category.
Apply: https://jobs.ashbyhq.com/cleric | Email: willem-hn@cleric.ai
Willem, Co-founder & CTO","created_at":"2026-02-02T18:30:24Z","created_at_i":1770057024,"objectID":"46859424","parent_id":46857488,"story_id":46857488,"story_title":"Ask HN: Who is hiring? (February 2026)","updated_at":"2026-03-05T23:29:28Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"gizajob"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"Well personally, as an investor I\u2019m more incredulous at the price. I hold both nvidia and apple and I\u2019ve made good money from both. It\u2019s worth looking at all the reasons for the price going up and what happens when the price goes down. So I\u2019m definitely not an nvidia hater because I think they\u2019re doing great things and clearly a brilliantly managed company, it\u2019s just that if the market cap gets too big then there\u2019s ultimately only one direction it can then go, and it\u2019s all predicated on an AI future that I\u2019m not certain is going to happen like it might in the best-case scenario given how we\u2019re constructing LLMs and how their corpuses of data are mined out already. My other main concern is towards how fickle human beings are - limitations with AIs are likely to cause a huge backlash once they become annoying or we feel like we\u2019re being fobbed off by corporates forcing us to interact with an AI agent that can only stick to a strict corporate line. It\u2019ll be boring, and people don\u2019t like being bored. I\u2019m not convinced that the current approaches to AI are going to deliver the goods that the graphs of progress are promising. And what\u2019s more is that it already seems like everyone who wants to use ChatGPT or Claude is able to use it, and in three years the hardware cost of running all that will have dropped by 4x, not run away as some nvidia-boosting exponential."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Nvidia is about to pass Apple in market cap"},"story_url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["ai"],"value":"https://www.reuters.com/technology/ai-darling-nvidias-market-value-surges-closer-apple-2024-05-28/"}},"_tags":["comment","author_gizajob","story_40515515"],"author":"gizajob","comment_text":"Well personally, as an investor I\u2019m more incredulous at the price. I hold both nvidia and apple and I\u2019ve made good money from both. It\u2019s worth looking at all the reasons for the price going up and what happens when the price goes down. So I\u2019m definitely not an nvidia hater because I think they\u2019re doing great things and clearly a brilliantly managed company, it\u2019s just that if the market cap gets too big then there\u2019s ultimately only one direction it can then go, and it\u2019s all predicated on an AI future that I\u2019m not certain is going to happen like it might in the best-case scenario given how we\u2019re constructing LLMs and how their corpuses of data are mined out already. My other main concern is towards how fickle human beings are - limitations with AIs are likely to cause a huge backlash once they become annoying or we feel like we\u2019re being fobbed off by corporates forcing us to interact with an AI agent that can only stick to a strict corporate line. It\u2019ll be boring, and people don\u2019t like being bored. I\u2019m not convinced that the current approaches to AI are going to deliver the goods that the graphs of progress are promising. And what\u2019s more is that it already seems like everyone who wants to use ChatGPT or Claude is able to use it, and in three years the hardware cost of running all that will have dropped by 4x, not run away as some nvidia-boosting exponential.","created_at":"2024-05-30T06:26:09Z","created_at_i":1717050369,"objectID":"40520705","parent_id":40519241,"story_id":40515515,"story_title":"Nvidia is about to pass Apple in market cap","story_url":"https://www.reuters.com/technology/ai-darling-nvidias-market-value-surges-closer-apple-2024-05-28/","updated_at":"2024-09-20T17:09:13Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"stego-tech"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"The big assumption with all of these sorts of analyses is that things will continue as they are for the foreseeable future.
In hype-driven markets, you cannot be certain of that.
Let's take a view that the author is right: coding agents and their associated harnesses were the inflection point for some degree of profitability and widespread consumption, and that these tools are now yet another SaaS subscription or API bucket expense to bake into every single developer (or developer-adjacent) in the organization alongside your collab suite, HR seat, CRM seat, design seat, etc. To be fair I honestly think that's a safe assumption to make for highly technical firms whose image is derived from remaining on the cutting edge of things.
That begs the following questions, which we won't know until IPOs start happening:
* Are subscriptions profitable, or just API consumption?
* What's the run rate when we just consider subscription-based usage like Claude Code and Codex? What about API calls?
* Is there any profitable pathway forward at which enterprises can get unlimited usage but at fixed rates via subscription?
* What does customer churn look like for subscription users versus API users?
We also have a number of questions for customers that I suspect we'll start seeing receipts for in the coming months, at least from the early adopters:
* What was the net gain (loss) from leveraging coding agents?
* What's the cost of a developer with or without access to a coding agent + harness? Is it cheaper to hire an outsourced worker with a coding agent subscription, or a domestic worker without one?
* At what point does further AI spend result in diminishing returns, i.e. where's the 'sweet spot' for spend?
* Did AI boost actual revenue and outcomes, or did it just gamify KPIs?
* What roles or work did AI actually replace, versus merely displace during the hype cycle?
Not to mention the questions regarding the technology itself:
* Will we develop the means to run foundational/frontier models at edge using less resources through some existing (e.g. distillation) or new technology, thus cutting off the profit centers of these firms?
* When the market mismatch between supply and demand is resolved, won't it be more affordable for consumers and companies to operate their own AI infrastructure rather than support further centralized buildouts?
* Will coding agents improve to the point of being able to bootstrap and self-orchestrate on edge/consumer hardware without substantial technical expertise, or at least improve to the point that traditional IT teams can securely operate them internally without an expensive subscription or API token bucket?
All of these will influence the long tail of this bubble, because it is a bubble at this point. Even if these companies are indeed profitable thanks to the coding agent inflection point, there's still so many unanswered questions about utility beyond coding that it's impossible to extrapolate a future. If coding agents are indeed the extent of utility for profitability, then there's no possible way these entities will recoup the investment already sunk into their infrastructure buildouts. Even if more profitable uses are discovered, does this offset or replace the firms disappearing due to AI speculation and their associated contributions to the economy as a whole (RE: the consumer compute industry at present, higher energy costs due to datacenter builds, opportunity cost from harms to local infrastructure from haphazard builds, etc)? Should these firms indeed be runaway successes and immensely profitable to the point of paying off their investors and growing the larger economy, does this end up stifling innovation in a world where most new ideas are fed into LLMs for R&D that are then controlled by only a handful of companies and immensely wealthy people, via systems that are easily surveilled and stolen from without recourse?
So many, many questions yet to be answered. Betting the farm because of coding agents is one hell of a gamble."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"I think Anthropic and OpenAI have found product-market fit"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://simonwillison.net/2026/May/27/product-market-fit/"}},"_tags":["comment","author_stego-tech","story_48296794"],"author":"stego-tech","comment_text":"The big assumption with all of these sorts of analyses is that things will continue as they are for the foreseeable future.
In hype-driven markets, you cannot be certain of that.
Let's take a view that the author is right: coding agents and their associated harnesses were the inflection point for some degree of profitability and widespread consumption, and that these tools are now yet another SaaS subscription or API bucket expense to bake into every single developer (or developer-adjacent) in the organization alongside your collab suite, HR seat, CRM seat, design seat, etc. To be fair I honestly think that's a safe assumption to make for highly technical firms whose image is derived from remaining on the cutting edge of things.
That begs the following questions, which we won't know until IPOs start happening:
* Are subscriptions profitable, or just API consumption?
* What's the run rate when we just consider subscription-based usage like Claude Code and Codex? What about API calls?
* Is there any profitable pathway forward at which enterprises can get unlimited usage but at fixed rates via subscription?
* What does customer churn look like for subscription users versus API users?
We also have a number of questions for customers that I suspect we'll start seeing receipts for in the coming months, at least from the early adopters:
* What was the net gain (loss) from leveraging coding agents?
* What's the cost of a developer with or without access to a coding agent + harness? Is it cheaper to hire an outsourced worker with a coding agent subscription, or a domestic worker without one?
* At what point does further AI spend result in diminishing returns, i.e. where's the 'sweet spot' for spend?
* Did AI boost actual revenue and outcomes, or did it just gamify KPIs?
* What roles or work did AI actually replace, versus merely displace during the hype cycle?
Not to mention the questions regarding the technology itself:
* Will we develop the means to run foundational/frontier models at edge using less resources through some existing (e.g. distillation) or new technology, thus cutting off the profit centers of these firms?
* When the market mismatch between supply and demand is resolved, won't it be more affordable for consumers and companies to operate their own AI infrastructure rather than support further centralized buildouts?
* Will coding agents improve to the point of being able to bootstrap and self-orchestrate on edge/consumer hardware without substantial technical expertise, or at least improve to the point that traditional IT teams can securely operate them internally without an expensive subscription or API token bucket?
All of these will influence the long tail of this bubble, because it is a bubble at this point. Even if these companies are indeed profitable thanks to the coding agent inflection point, there's still so many unanswered questions about utility beyond coding that it's impossible to extrapolate a future. If coding agents are indeed the extent of utility for profitability, then there's no possible way these entities will recoup the investment already sunk into their infrastructure buildouts. Even if more profitable uses are discovered, does this offset or replace the firms disappearing due to AI speculation and their associated contributions to the economy as a whole (RE: the consumer compute industry at present, higher energy costs due to datacenter builds, opportunity cost from harms to local infrastructure from haphazard builds, etc)? Should these firms indeed be runaway successes and immensely profitable to the point of paying off their investors and growing the larger economy, does this end up stifling innovation in a world where most new ideas are fed into LLMs for R&D that are then controlled by only a handful of companies and immensely wealthy people, via systems that are easily surveilled and stolen from without recourse?
So many, many questions yet to be answered. Betting the farm because of coding agents is one hell of a gamble.","created_at":"2026-05-27T18:57:22Z","created_at_i":1779908242,"objectID":"48298834","parent_id":48296794,"story_id":48296794,"story_title":"I think Anthropic and OpenAI have found product-market fit","story_url":"https://simonwillison.net/2026/May/27/product-market-fit/","updated_at":"2026-05-28T14:56:34Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"gamechangr"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"Mark Zuckerbergs comment for quick reference: (link in the article)
Every new year of the last decade I set a personal challenge. My goal was to grow in new ways outside my day-to-day work running Facebook. These led me to learn Mandarin, code an AI assistant for my home, read more books, run a lot more, learn to hunt and cook, and get more comfortable with public speaking.
When I started these challenges, my life was almost all about building the Facebook website. (It was mostly a website at the time.) Now there's so much more to learn from. At Facebook, we're building lots of different apps and technology -- ranging from a new private social platform to augmented and virtual reality -- and we're handling a lot more social responsibility. And outside Facebook, I'm a father now and I love spending time with my family, working on our philanthropy, and improving at the sports and hobbies I've picked up over the years. So while I'm glad I did annual challenges over the last decade, it's time to do something different.
This decade I'm going to take a longer term focus. Rather than having year-to-year challenges, I've tried to think about what I hope the world and my life will look in 2030 so I can make sure I'm focusing on those things. By then, if things go well, my daughter Max will be in high school, we'll have the technology to feel truly present with another person no matter where they are, and scientific research will have helped cure and prevent enough diseases to extend our average life expectancy by another 2.5 years. Here are some of the things that I think will be important in the next decade:
---\nGenerational Change
When I started Facebook, one of the reasons I cared about giving people a voice was that I thought it would empower my generation -- which I felt had important things to say and weren't being listened to enough. It turned out it wasn't just my generation that felt marginalized and needed more voice though, and these tools have given power to lots of different groups across society. I'm glad more people have voice, but it hasn't yet brought about the generational change in addressing important issues I had hoped for. I think that will happen this decade.
Today, many important institutions in our society still aren't doing enough to address the issues younger generations face -- from climate change to runaway costs of education, housing and healthcare. But as millennials and more members of younger generations can vote, I expect this to start changing rapidly. By the end of this decade, I expect more institutions will be run by millennials and more policies will be set to address these problems with longer term outlooks.
In many ways, Facebook is a millennial company with the issues of this generation in mind. At the Chan Zuckerberg Initiative, our focus is on very long term efforts that will primarily help our children's generation, like investing in curing, preventing and managing all diseases in our children's lifetimes, or making primary education more personalized to students needs. Over the next decade, we'll focus more on funding and giving a platform to younger entrepreneurs, scientists, and leaders to enable these changes.\n---\nA New Private Social Platform
The internet gave us the superpower of being able to connect with anyone, anywhere. This is incredibly empowering and means that our relationships and opportunities are no longer confined to just where we live. We're now part of a community with billions of people in it -- with all the dynamism, culture and economic opportunity that brings.
But being part of such a large community creates its own challenges and makes us crave intimacy. When I grew up in a small town, it was easy to have a niche and sense of purpose. But with billions of people, it's harder to find your unique role. For the next decade, some of the most important social infrastructure will help us reconstruct all kinds of smaller communities to give us that sense of intimacy again.
This is one of the areas of innovation I'm most excited about. Our digital social environments will feel very different over the next 5+ years, re-emphasizing private interactions and helping us build the smaller communities we all need in our lives.\n---
Decentralizing Opportunity
In the last decade, the fastest growth in the economy has been in the tech industry. In the next decade, I expect technology will continue to create opportunity, but more through enabling all of the other parts of the economy to make better use of technology and grow even faster. \nThe area we're most focused on is helping small businesses. Across our services, more than 140 million small businesses already reach customers -- mostly for free. Today this takes the form of an entrepreneur setting up an account on Facebook, Instagram, or WhatsApp, and then either communicating with people for free or buying ads to get their message out more broadly. Over the next decade, we hope to build the commerce and payments tools so that every small business has easy access to the same technology that previously only big companies have had.
If we can make it so anyone can sell products through a storefront on Instagram, message and support their customers through Messenger, or send money home to another country instantly and at low cost through WhatsApp -- that will go a long way towards creating more opportunity around the world. At the end of the day, a strong and stable economy comes from people succeeding broadly, and the best way to do that is to make it so small businesses can effectively become technology companies.
---
The Next Computing Platform
The technology platform of the 2010s was the mobile phone. The platform of the 2000s before that was about the web, and the 1990s was the desktop computer. Each computing platform becomes more ubiquitously accessible and natural for us to interact with. While I expect phones to still be our primary devices through most of this decade, at some point in the 2020s, we will get breakthrough augmented reality glasses that will redefine our relationship with technology.
Augmented and virtual reality are about delivering a sense of presence -- the feeling that you're right there with another person or in another place. Instead of having devices that take us away from the people around us, the next platform will help us be more present with each other and will help the technology get out of the way. Even though some of the early devices seem clunky, I think these will be the most human and social technology platforms anyone has built yet. \nThe ability to be "present" anywhere will also help us address some of the biggest social issues of our day -- like ballooning housing costs and inequality of opportunity by geography. Today, many people feel like they have to move to cities because that's where the jobs are. But there isn't enough housing in many cities, so housing costs are skyrocketing while quality of living is decreasing. Imagine if you could live anywhere you chose and access any job anywhere else. If we deliver on what we're building, this should be much closer to reality by 2030.
---
New Forms of Governance
One of the big questions for the next decade is: how should we govern the large new digital communities that the internet has enabled? Platforms like Facebook have to make tradeoffs on social values we all hold dear -- like between free expression and safety, or between privacy and law enforcement, or between creating open systems and locking down data and access. It's rare that there's ever a clear "right" answer, and in many cases it's as important that the decisions are made in a way that feels legitimate to the community. From this perspective, I don't think private companies should be making so many important decisions that touch on fundamental democratic values.
One way to address this is through regulation. As long as our governments are seen as legitimate, rules established through a democratic process could add more legitimacy and trust than rules defined by companies alone. There are a number of areas where I believe governments establishing clearer rules would be helpful, including around elections, harmful content, privacy, and data portability. I've called for new regulation in these areas and over the next decade I hope we get clearer rules for the internet.
Another and perhaps even better way to address this is by establishing new ways for communities to govern themselves. An example of independent governance is the Oversight Board we're creating. Soon you'll be able to appeal content decisions you disagree with to an independent board that will have the final decision in whether something is allowed. This decade, I hope to use my position to establish more community governance and more institutions like this. If this is successful, it could be a model for other online communities in the future.
---
We've got a lot to do this decade and there's a lot to learn to help make this all happen. I hope your new year and new decade are off to a good start. Here's to a great 2020s."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Zuckerberg drops personal challenges in favor of big-picture plans for thedecade"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.theguardian.com/technology/2020/jan/09/mark-zuckerberg-annual-personal-challenge"}},"_tags":["comment","author_gamechangr","story_25524593"],"author":"gamechangr","comment_text":"Mark Zuckerbergs comment for quick reference: (link in the article)
Every new year of the last decade I set a personal challenge. My goal was to grow in new ways outside my day-to-day work running Facebook. These led me to learn Mandarin, code an AI assistant for my home, read more books, run a lot more, learn to hunt and cook, and get more comfortable with public speaking.
When I started these challenges, my life was almost all about building the Facebook website. (It was mostly a website at the time.) Now there's so much more to learn from. At Facebook, we're building lots of different apps and technology -- ranging from a new private social platform to augmented and virtual reality -- and we're handling a lot more social responsibility. And outside Facebook, I'm a father now and I love spending time with my family, working on our philanthropy, and improving at the sports and hobbies I've picked up over the years. So while I'm glad I did annual challenges over the last decade, it's time to do something different.
This decade I'm going to take a longer term focus. Rather than having year-to-year challenges, I've tried to think about what I hope the world and my life will look in 2030 so I can make sure I'm focusing on those things. By then, if things go well, my daughter Max will be in high school, we'll have the technology to feel truly present with another person no matter where they are, and scientific research will have helped cure and prevent enough diseases to extend our average life expectancy by another 2.5 years. Here are some of the things that I think will be important in the next decade:
---\nGenerational Change
When I started Facebook, one of the reasons I cared about giving people a voice was that I thought it would empower my generation -- which I felt had important things to say and weren't being listened to enough. It turned out it wasn't just my generation that felt marginalized and needed more voice though, and these tools have given power to lots of different groups across society. I'm glad more people have voice, but it hasn't yet brought about the generational change in addressing important issues I had hoped for. I think that will happen this decade.
Today, many important institutions in our society still aren't doing enough to address the issues younger generations face -- from climate change to runaway costs of education, housing and healthcare. But as millennials and more members of younger generations can vote, I expect this to start changing rapidly. By the end of this decade, I expect more institutions will be run by millennials and more policies will be set to address these problems with longer term outlooks.
In many ways, Facebook is a millennial company with the issues of this generation in mind. At the Chan Zuckerberg Initiative, our focus is on very long term efforts that will primarily help our children's generation, like investing in curing, preventing and managing all diseases in our children's lifetimes, or making primary education more personalized to students needs. Over the next decade, we'll focus more on funding and giving a platform to younger entrepreneurs, scientists, and leaders to enable these changes.\n---\nA New Private Social Platform
The internet gave us the superpower of being able to connect with anyone, anywhere. This is incredibly empowering and means that our relationships and opportunities are no longer confined to just where we live. We're now part of a community with billions of people in it -- with all the dynamism, culture and economic opportunity that brings.
But being part of such a large community creates its own challenges and makes us crave intimacy. When I grew up in a small town, it was easy to have a niche and sense of purpose. But with billions of people, it's harder to find your unique role. For the next decade, some of the most important social infrastructure will help us reconstruct all kinds of smaller communities to give us that sense of intimacy again.
This is one of the areas of innovation I'm most excited about. Our digital social environments will feel very different over the next 5+ years, re-emphasizing private interactions and helping us build the smaller communities we all need in our lives.\n---
Decentralizing Opportunity
In the last decade, the fastest growth in the economy has been in the tech industry. In the next decade, I expect technology will continue to create opportunity, but more through enabling all of the other parts of the economy to make better use of technology and grow even faster. \nThe area we're most focused on is helping small businesses. Across our services, more than 140 million small businesses already reach customers -- mostly for free. Today this takes the form of an entrepreneur setting up an account on Facebook, Instagram, or WhatsApp, and then either communicating with people for free or buying ads to get their message out more broadly. Over the next decade, we hope to build the commerce and payments tools so that every small business has easy access to the same technology that previously only big companies have had.
If we can make it so anyone can sell products through a storefront on Instagram, message and support their customers through Messenger, or send money home to another country instantly and at low cost through WhatsApp -- that will go a long way towards creating more opportunity around the world. At the end of the day, a strong and stable economy comes from people succeeding broadly, and the best way to do that is to make it so small businesses can effectively become technology companies.
---
The Next Computing Platform
The technology platform of the 2010s was the mobile phone. The platform of the 2000s before that was about the web, and the 1990s was the desktop computer. Each computing platform becomes more ubiquitously accessible and natural for us to interact with. While I expect phones to still be our primary devices through most of this decade, at some point in the 2020s, we will get breakthrough augmented reality glasses that will redefine our relationship with technology.
Augmented and virtual reality are about delivering a sense of presence -- the feeling that you're right there with another person or in another place. Instead of having devices that take us away from the people around us, the next platform will help us be more present with each other and will help the technology get out of the way. Even though some of the early devices seem clunky, I think these will be the most human and social technology platforms anyone has built yet. \nThe ability to be "present" anywhere will also help us address some of the biggest social issues of our day -- like ballooning housing costs and inequality of opportunity by geography. Today, many people feel like they have to move to cities because that's where the jobs are. But there isn't enough housing in many cities, so housing costs are skyrocketing while quality of living is decreasing. Imagine if you could live anywhere you chose and access any job anywhere else. If we deliver on what we're building, this should be much closer to reality by 2030.
---
New Forms of Governance
One of the big questions for the next decade is: how should we govern the large new digital communities that the internet has enabled? Platforms like Facebook have to make tradeoffs on social values we all hold dear -- like between free expression and safety, or between privacy and law enforcement, or between creating open systems and locking down data and access. It's rare that there's ever a clear "right" answer, and in many cases it's as important that the decisions are made in a way that feels legitimate to the community. From this perspective, I don't think private companies should be making so many important decisions that touch on fundamental democratic values.
One way to address this is through regulation. As long as our governments are seen as legitimate, rules established through a democratic process could add more legitimacy and trust than rules defined by companies alone. There are a number of areas where I believe governments establishing clearer rules would be helpful, including around elections, harmful content, privacy, and data portability. I've called for new regulation in these areas and over the next decade I hope we get clearer rules for the internet.
Another and perhaps even better way to address this is by establishing new ways for communities to govern themselves. An example of independent governance is the Oversight Board we're creating. Soon you'll be able to appeal content decisions you disagree with to an independent board that will have the final decision in whether something is allowed. This decade, I hope to use my position to establish more community governance and more institutions like this. If this is successful, it could be a model for other online communities in the future.
---
We've got a lot to do this decade and there's a lot to learn to help make this all happen. I hope your new year and new decade are off to a good start. Here's to a great 2020s.","created_at":"2020-12-24T02:46:00Z","created_at_i":1608777960,"objectID":"25524610","parent_id":25524593,"story_id":25524593,"story_title":"Zuckerberg drops personal challenges in favor of big-picture plans for thedecade","story_url":"https://www.theguardian.com/technology/2020/jan/09/mark-zuckerberg-annual-personal-challenge","updated_at":"2024-09-20T07:35:19Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"reissbaker"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"The costs for OpenAI and Google aren't public, but if you look at the open-source models, inference is very cheap: for example, you can generally beat the public serverless prices by a factor of ~2 by using dedicated GPUs [1], and given that a 70b model costs about $1/million tokens serverless \u2014 and tend to perform similarly on benchmarks to 4o \u2014 OpenAI is most likely getting very fat profit margins at $2.50/million input tokens and $10/million output tokens.
The problem for them is making enough money for the training runs (where it seems like their strategy is to raise money on the hope they achieve some kind of runaway self-improving effect that grants them an effective monopoly on the leading models, combined with regulatory pushes to ban their competitors) \u2014 but it seems very unlikely to me that they're losing money serving the models.
1: https://fireworks.ai/blog/why-gpus-on-demand"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Operator research preview"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://openai.com/index/introducing-operator/"}},"_tags":["comment","author_reissbaker","story_42806301"],"author":"reissbaker","comment_text":"The costs for OpenAI and Google aren't public, but if you look at the open-source models, inference is very cheap: for example, you can generally beat the public serverless prices by a factor of ~2 by using dedicated GPUs [1], and given that a 70b model costs about $1/million tokens serverless \u2014 and tend to perform similarly on benchmarks to 4o \u2014 OpenAI is most likely getting very fat profit margins at $2.50/million input tokens and $10/million output tokens.
The problem for them is making enough money for the training runs (where it seems like their strategy is to raise money on the hope they achieve some kind of runaway self-improving effect that grants them an effective monopoly on the leading models, combined with regulatory pushes to ban their competitors) \u2014 but it seems very unlikely to me that they're losing money serving the models.
1: https://fireworks.ai/blog/why-gpus-on-demand","created_at":"2025-01-24T02:41:21Z","created_at_i":1737686481,"objectID":"42810042","parent_id":42808788,"story_id":42806301,"story_title":"Operator research preview","story_url":"https://openai.com/index/introducing-operator/","updated_at":"2025-01-24T17:55:45Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"za_mike157"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"Hi HN,
We are Michael & Jono, and we are building Cerebrium (https://www.cerebrium.ai), a serverless infrastructure platform for ML/AI applications - we make it easy for engineers to build, deploy and scale AI applications.
Initially, we\u2019ve been hyper-focused on the inference side of applications, but we\u2019re working on expanding our functionality to support training and data processing use cases\u2014eventually covering the full AI development lifecycle.
You can watch a quick loom video of us deploying: https://www.loom.com/share/06947794b3bf4bb1bb21c87066dfcc66?...
How we got here:
Jono and I led the technical team at our previous e-commerce startup, which grew rapidly over a few years. As we scaled, we were tasked with building out ML applications to make the business more efficient. It was tough\u2014every day felt like a defeat. We found ourselves stitching together AWS Lambda, Sagemaker, and Prefect jobs (this stack alone was enough to make me want to give up). By the time we reached production, the costs were too high to maintain. Getting these applications live required a significant upfront investment of both time and money, making it inaccessible for most startups and scale-ups to attempt. We wanted to create something that would help us (and others like us) implement ML/AI applications easily and cost-effectively.
The problem:
There are a ton of challenges to tackle to realize our vision, but we\u2019ve initially focused on a few key ones:
1. GPUs are expensive \u2013 An A100 is 326 times the cost of a CPU, and companies are using LLMs like they\u2019re simple APIs. Serverless instances solve this to an extent, but minimizing cold starts is difficult.
2. Local development \u2013 Engineers need local development environments to iterate quickly, but production-grade GPUs aren\u2019t available on consumer hardware. How can we make cloud deployments feel as fast as just saving a file locally and retrying?
3. Cost to experiment - To run experiments we had to spin up EC2 instances each day, recreate our environment and run scripts. It was difficult to monitor logs, instance usage metrics as well as run large processing jobs or scale endpoints without a significant infrastructure investment. Additionally, we often forget to switch off instances which cost us money!
Our Approach
We have three core areas that we are focused on which we believe are the most important for any infrastructure platform:
1. Performance:
We have worked hard to get our added network latency <50ms and the cold start of our average workloads to 2-4 seconds. Here are a few things we did to get our cold starts so low:
- Container Runtime: We built our own container runtime that splits container images into two parts\u2014metadata and data blobs. Metadata provides the file structure, while the actual data blobs are fetched on-demand. This allows containers to start before the full image is downloaded. In the background, we prefetch the remaining blobs.
- Caching: Once an image is on a machine, it\u2019s cached for future use. This makes subsequent container startups much faster. We also intelligently route requests to machines where the image is already cached.
- Efficient Inference: We route requests to the optimal machines, prioritizing low-latency and high-throughput performance. If no containers are immediately available, we efficiently queue the requests through our task scheduling system.
- Distributed Storage Cache: One of the most resource-intensive parts of AI workloads is loading models into VRAM. We use NVME drives (which are much faster than network volumes), as close as possible to the machines and we orchestrate workloads to nodes that already contain the necessary model weights where possible.
2. Developer Experience
We built Cerebrium to help developers iterate as quickly as possible by streamlining the entire build and deployment process.
To get build times as low as possible, we use high-performance machines and cache layers where possible. We've reduced first-time build times to an average of 2 minutes and 24 seconds, with subsequent builds completing in just 19 seconds.
We also offer a wide range of GPU types\u2014over 8 different options\u2014so you can easily test performance and cost efficiency by adjusting a single line in your configuration file.
To reduce friction, we\u2019ve kept things simple. There are no custom Python decorators, no Cerebrium specific syntax to learn. You just add a .toml file to define your hardware requirements and environment settings. This makes migrating onto or off our platform just as easy as migrating off. We aim to impress you enough that you will want to stay.
3. Stability
This is arguably more important than the first two areas - no one wants to get an email at 11pm at night or on a Saturday that their application is down or degraded. Since April, we\u2019ve maintained 99.999% uptime. We have redundancies in place, monitoring, alerts, and a team that covers all time zones to resolve any issues quickly.
Why Is This Hard?
Building Cerebrium has been challenging because it involves solving multiple interconnected problems. It requires optimization at every step\u2014from efficiently splitting images to fetching data on-demand without introducing latency, handling distributed caching, optimizing our network stack, and ensuring redundancies, all while holding true to the three areas mentioned above.
Pricing:
We charge you exactly for the resources you need and only charge you when your code is running ie: usage-based. For example, if you specify you need 1 A100 GPU, with 2 CPUs and 12 GB of RAM we charge you exactly for that and not a full A100 (12 CPU\u2019s and 148GB of memory)
You can see more about our pricing here: http://www.cerebrium.ai/pricing
What\u2019s Next?
We're builders too, and we know how crucial support can be when you're working on something new. Here's what we've put together to support teams like yours:
- $30 in free credit to start exploring. If you're onto something interesting but need more runway, just give us a shout - we\u2019d be happy to extend that for compelling use-cases.
- We have worked hard on our docs to make onboarding easy as well as have a very elaborate Github repo covering AI voice agents, LLM optimizations and much more.
Docs: https://docs.cerebrium.ai/cerebrium/getting-started/introduc...
Github Examples: https://github.com/CerebriumAI/examples/tree/master
If you have a question or hit a snag, you can directly reach out to the engineers who built the platform\u2014we\u2019re here to help! We\u2019ve also set up Slack and Discord communities where you can connect with other creators, share experiences, ask for advice, or just chat with folks building cool things.
We're looking forward to seeing what you all build and please give us feedback on what you would like us to improve/add"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["ai"],"value":"Launch HN: Cerebrium (YC W22) \u2013 Serverless Infrastructure Platform for ML/AI"}},"_tags":["story","author_za_mike157","story_41579777","launch_hn"],"author":"za_mike157","children":[41581063,41581606,41581641,41581978,41582412,41583034,41583567,41585296,41585638,41586731,41588056,41588178,41589141],"created_at":"2024-09-18T13:54:54Z","created_at_i":1726667694,"num_comments":32,"objectID":"41579777","points":48,"story_id":41579777,"story_text":"Hi HN,
We are Michael & Jono, and we are building Cerebrium (https://www.cerebrium.ai), a serverless infrastructure platform for ML/AI applications - we make it easy for engineers to build, deploy and scale AI applications.
Initially, we\u2019ve been hyper-focused on the inference side of applications, but we\u2019re working on expanding our functionality to support training and data processing use cases\u2014eventually covering the full AI development lifecycle.
You can watch a quick loom video of us deploying: https://www.loom.com/share/06947794b3bf4bb1bb21c87066dfcc66?...
How we got here:
Jono and I led the technical team at our previous e-commerce startup, which grew rapidly over a few years. As we scaled, we were tasked with building out ML applications to make the business more efficient. It was tough\u2014every day felt like a defeat. We found ourselves stitching together AWS Lambda, Sagemaker, and Prefect jobs (this stack alone was enough to make me want to give up). By the time we reached production, the costs were too high to maintain. Getting these applications live required a significant upfront investment of both time and money, making it inaccessible for most startups and scale-ups to attempt. We wanted to create something that would help us (and others like us) implement ML/AI applications easily and cost-effectively.
The problem:
There are a ton of challenges to tackle to realize our vision, but we\u2019ve initially focused on a few key ones:
1. GPUs are expensive \u2013 An A100 is 326 times the cost of a CPU, and companies are using LLMs like they\u2019re simple APIs. Serverless instances solve this to an extent, but minimizing cold starts is difficult.
2. Local development \u2013 Engineers need local development environments to iterate quickly, but production-grade GPUs aren\u2019t available on consumer hardware. How can we make cloud deployments feel as fast as just saving a file locally and retrying?
3. Cost to experiment - To run experiments we had to spin up EC2 instances each day, recreate our environment and run scripts. It was difficult to monitor logs, instance usage metrics as well as run large processing jobs or scale endpoints without a significant infrastructure investment. Additionally, we often forget to switch off instances which cost us money!
Our Approach
We have three core areas that we are focused on which we believe are the most important for any infrastructure platform:
1. Performance:
We have worked hard to get our added network latency <50ms and the cold start of our average workloads to 2-4 seconds. Here are a few things we did to get our cold starts so low:
- Container Runtime: We built our own container runtime that splits container images into two parts\u2014metadata and data blobs. Metadata provides the file structure, while the actual data blobs are fetched on-demand. This allows containers to start before the full image is downloaded. In the background, we prefetch the remaining blobs.
- Caching: Once an image is on a machine, it\u2019s cached for future use. This makes subsequent container startups much faster. We also intelligently route requests to machines where the image is already cached.
- Efficient Inference: We route requests to the optimal machines, prioritizing low-latency and high-throughput performance. If no containers are immediately available, we efficiently queue the requests through our task scheduling system.
- Distributed Storage Cache: One of the most resource-intensive parts of AI workloads is loading models into VRAM. We use NVME drives (which are much faster than network volumes), as close as possible to the machines and we orchestrate workloads to nodes that already contain the necessary model weights where possible.
2. Developer Experience
We built Cerebrium to help developers iterate as quickly as possible by streamlining the entire build and deployment process.
To get build times as low as possible, we use high-performance machines and cache layers where possible. We've reduced first-time build times to an average of 2 minutes and 24 seconds, with subsequent builds completing in just 19 seconds.
We also offer a wide range of GPU types\u2014over 8 different options\u2014so you can easily test performance and cost efficiency by adjusting a single line in your configuration file.
To reduce friction, we\u2019ve kept things simple. There are no custom Python decorators, no Cerebrium specific syntax to learn. You just add a .toml file to define your hardware requirements and environment settings. This makes migrating onto or off our platform just as easy as migrating off. We aim to impress you enough that you will want to stay.
3. Stability
This is arguably more important than the first two areas - no one wants to get an email at 11pm at night or on a Saturday that their application is down or degraded. Since April, we\u2019ve maintained 99.999% uptime. We have redundancies in place, monitoring, alerts, and a team that covers all time zones to resolve any issues quickly.
Why Is This Hard?
Building Cerebrium has been challenging because it involves solving multiple interconnected problems. It requires optimization at every step\u2014from efficiently splitting images to fetching data on-demand without introducing latency, handling distributed caching, optimizing our network stack, and ensuring redundancies, all while holding true to the three areas mentioned above.
Pricing:
We charge you exactly for the resources you need and only charge you when your code is running ie: usage-based. For example, if you specify you need 1 A100 GPU, with 2 CPUs and 12 GB of RAM we charge you exactly for that and not a full A100 (12 CPU\u2019s and 148GB of memory)
You can see more about our pricing here: http://www.cerebrium.ai/pricing
What\u2019s Next?
We're builders too, and we know how crucial support can be when you're working on something new. Here's what we've put together to support teams like yours:
- $30 in free credit to start exploring. If you're onto something interesting but need more runway, just give us a shout - we\u2019d be happy to extend that for compelling use-cases.
- We have worked hard on our docs to make onboarding easy as well as have a very elaborate Github repo covering AI voice agents, LLM optimizations and much more.
Docs: https://docs.cerebrium.ai/cerebrium/getting-started/introduc...
Github Examples: https://github.com/CerebriumAI/examples/tree/master
If you have a question or hit a snag, you can directly reach out to the engineers who built the platform\u2014we\u2019re here to help! We\u2019ve also set up Slack and Discord communities where you can connect with other creators, share experiences, ask for advice, or just chat with folks building cool things.
We're looking forward to seeing what you all build and please give us feedback on what you would like us to improve/add","title":"Launch HN: Cerebrium (YC W22) \u2013 Serverless Infrastructure Platform for ML/AI","updated_at":"2024-10-05T21:16:13Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"silenced_sage"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"The problem isn't politics, its been made to look that way but the issue is that the people who would know have been silenced in a way that the general public hasn't recognized. Its not in the benefit of the corporate overlords that you be properly educated on this.
I'll give you a brief TL;DR on the issues.
The wealth of a nation is based in its ability to produce primarily goods, and sometimes services. Pricing is a signaling mechanism that allows efficient resource allocation of both goods, and factor market labor.
When misallocation results, you get things like busts and booms where the benefits are front-loaded and a resource exhaustion cycle takes place. There are constraints which cannot be breached for any length of time, such as wages being lower than the cost of living enough to support a wife and three children to 18, and other things. Legitimate business can't operate if it can't make a profit.
There is silently nationalized industry that is unconstrained via a cycle of money-printing that has sieved assets into few hands, and risk concentrated in few hands. This is part of the danger of the ECP, and other fundamental failures related to centralized systems.
The Austrian school lives in the future. The future under a breakdown of organized society is one of death and potentially extinction. We depend on food production in ecological overshoot which depends on technology and farming. The 1970s represented ecological overshoot. When that fails you get shortage, that then sustains, famine, slavery, death, and then extinction.
Under socio-economic collapse everything fails. Currency is abandoned. Exchange cannot happen. The store of value and exchange of value fails. This happened during the Bronze Age (of which we have very little records that remain).
If these happen for extended periods of time, capital dries up and you get something like the great depression, Nintendo, and other places where logistics refuses to deliver goods at any potential profit because they were burned too many times.
There exist systems that will fail eventually and predictably (in general), but which you cannot predict specifically when those failures will happen, its unknowable, and the effects one would use to justify any decision lag behind the objective indicators of the actions that needed to be avoided.
Money-printing extracts value from those that hold the currency through the cantillion effect. Its an extraction of unpaid slave labor. Eventually it exceeds the store of value, and the businesses that normally produce stop producing. Exchange stops, etc. AI presents a unique spin on this as well because it accelerates the corruption of signalling presenting false signals chaotically in both labor and good markets.
In positive feedback systems, these type of systems commonly run-away and converge at a point of calamity. When they do the dynamics are unstoppable.
The MMT folks are delusional, having rested quite a lot of their false justifications on unsound practice to justify extraction of value.
The Keynesian's aren't so different from the MMT folks because they improperly look at certain things only in isolation with a seeming mental block to all other things, which is a form of false justification.
The bankers (if you can call them that) have corrupted leadership, and enabled sieving through non-reserve debt issuance. The mechanism over the years is leveraged buyout. The business cycle today largely runs on the ponzi cycle with debt issued upfront providing benefits upfront, followed by enshittification as the resource exhaustion cycle runs its course.
Chaos is fundamentally destructive. Large societies don't implode when they have stable stores of value, a rule of law (not by law), and manufacture goods their populace needs.
Market dynamics fundamentally fail to operate as a market under slave labor as a function of the cost function for the majority of participants. It doesn't matter if that labor is provided by a foreign power with costs socialized, or stolen through the currency through deficit spending. When you cannot know correct prices, everything falls apart at fundamental levels but you don't see it until its too late after which point the only thing you can do is start over; but existing structures under such systems seek control to the point of extinction.
No one gives power up willingly, and those that have it seek to destroy the ability for others to take it.
You can't ever make a consensus when a good portion of the people part of that consensus have become delusional, often without them even realizing it."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"U.S. Debt Tops 100% of GDP"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.wsj.com/economy/u-s-debt-tops-100-of-gdp-81c013d7"}},"_tags":["comment","author_silenced_sage","story_47961804"],"author":"silenced_sage","comment_text":"The problem isn't politics, its been made to look that way but the issue is that the people who would know have been silenced in a way that the general public hasn't recognized. Its not in the benefit of the corporate overlords that you be properly educated on this.
I'll give you a brief TL;DR on the issues.
The wealth of a nation is based in its ability to produce primarily goods, and sometimes services. Pricing is a signaling mechanism that allows efficient resource allocation of both goods, and factor market labor.
When misallocation results, you get things like busts and booms where the benefits are front-loaded and a resource exhaustion cycle takes place. There are constraints which cannot be breached for any length of time, such as wages being lower than the cost of living enough to support a wife and three children to 18, and other things. Legitimate business can't operate if it can't make a profit.
There is silently nationalized industry that is unconstrained via a cycle of money-printing that has sieved assets into few hands, and risk concentrated in few hands. This is part of the danger of the ECP, and other fundamental failures related to centralized systems.
The Austrian school lives in the future. The future under a breakdown of organized society is one of death and potentially extinction. We depend on food production in ecological overshoot which depends on technology and farming. The 1970s represented ecological overshoot. When that fails you get shortage, that then sustains, famine, slavery, death, and then extinction.
Under socio-economic collapse everything fails. Currency is abandoned. Exchange cannot happen. The store of value and exchange of value fails. This happened during the Bronze Age (of which we have very little records that remain).
If these happen for extended periods of time, capital dries up and you get something like the great depression, Nintendo, and other places where logistics refuses to deliver goods at any potential profit because they were burned too many times.
There exist systems that will fail eventually and predictably (in general), but which you cannot predict specifically when those failures will happen, its unknowable, and the effects one would use to justify any decision lag behind the objective indicators of the actions that needed to be avoided.
Money-printing extracts value from those that hold the currency through the cantillion effect. Its an extraction of unpaid slave labor. Eventually it exceeds the store of value, and the businesses that normally produce stop producing. Exchange stops, etc. AI presents a unique spin on this as well because it accelerates the corruption of signalling presenting false signals chaotically in both labor and good markets.
In positive feedback systems, these type of systems commonly run-away and converge at a point of calamity. When they do the dynamics are unstoppable.
The MMT folks are delusional, having rested quite a lot of their false justifications on unsound practice to justify extraction of value.
The Keynesian's aren't so different from the MMT folks because they improperly look at certain things only in isolation with a seeming mental block to all other things, which is a form of false justification.
The bankers (if you can call them that) have corrupted leadership, and enabled sieving through non-reserve debt issuance. The mechanism over the years is leveraged buyout. The business cycle today largely runs on the ponzi cycle with debt issued upfront providing benefits upfront, followed by enshittification as the resource exhaustion cycle runs its course.
Chaos is fundamentally destructive. Large societies don't implode when they have stable stores of value, a rule of law (not by law), and manufacture goods their populace needs.
Market dynamics fundamentally fail to operate as a market under slave labor as a function of the cost function for the majority of participants. It doesn't matter if that labor is provided by a foreign power with costs socialized, or stolen through the currency through deficit spending. When you cannot know correct prices, everything falls apart at fundamental levels but you don't see it until its too late after which point the only thing you can do is start over; but existing structures under such systems seek control to the point of extinction.
No one gives power up willingly, and those that have it seek to destroy the ability for others to take it.
You can't ever make a consensus when a good portion of the people part of that consensus have become delusional, often without them even realizing it.","created_at":"2026-04-30T14:05:22Z","created_at_i":1777557922,"objectID":"47962714","parent_id":47962135,"story_id":47961804,"story_title":"U.S. Debt Tops 100% of GDP","story_url":"https://www.wsj.com/economy/u-s-debt-tops-100-of-gdp-81c013d7","updated_at":"2026-04-30T15:24:53Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"echelon"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ai","agent","cost","runaway","loop"],"value":"I feel like we're dancing on the razor's edge.
On the one hand: high inflation, tariffs, layoffs, unemployment, high interest rate, energy crisis. Tons of economic red flags flashing.
On the other hand: AI is showing signs of being the next industrial revolution, we're re-industrializing, onshoring/friendshoring, and have a clear lead on chips and space tech at a time when it matters the most.
It's absolutely insane that Claude Code can spit out a week's worth of business automation tasks in half a day. And do it at relatively high quality in low-defect rate languages like Rust.
Europe won't be able to catch that. They're too busy regulating ahead of the tech. They're going to be a decade behind if they keep it up.
If we cut the chip supply right as things take off, China might not either. In a runaway takeoff scenario, they replace all their factory workers with robots and quickly scale and cost optimize. If America is smart, we might be able to do that too.
Our growth could accelerate or crater. These are wild times. More exciting than the last 20.
America needs to start pumping out new energy projects. It needs to make friends with all of its former allies. And it needs to import PhD students.
And we do need factories and raw inputs. The robots will take over for humans within a decade. If we stick the landing, we could be the new China right here at home.
Edit: rate limited on replies, so updating my comment instead.
Edit 2: Europe supplies the EUV lithography, but intelligence manifests higher up the stack. If we're talking rate limits, lots of countries supply critical inputs. \nI'm saying that Europe hasn't made strides towards developing their own models and infra, and it doesn't look like it's even close to starting to attack this problem. I want it to.
Edit 3: What I'm saying is that these tailwinds might put America back into the position it was in post-WWII. Manufacturing, tech, and science powerhouse in all the places that matter. Peers a generation or two behind. That's literally where America was after the war, and it looks like we could be teeing up for a repeat if it all doesn't unravel first.
America needs to double down on investing in energy and factories now. It looks like it will pay off in a big way.
Edit 4:
> You think Europe won't be able to use Claude Code
I would be extremely geopolitically anxious to rely on another country's tech in a take off scenario. Those tokens might be diverted to US businesses and factories. Or the US might strong arm concessions out of Europe. Europe needs domestic capability for this now.
It's not just Europe and sovereign nations. Workers and labor capital will be effectively frozen out of participation if there aren't open source equivalents.
> This is an downright evil take on the current situation.
It's just reality. Multipolarity means we're going to see a lot more of this type of framing, because it's what's happening on the ground."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What happens when US economic data becomes unreliable"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://mitsloan.mit.edu/ideas-made-to-matter/what-happens-when-us-economic-data-becomes-unreliable"}},"_tags":["comment","author_echelon","story_47378638"],"author":"echelon","children":[47378976,47378996,47379039,47379123,47379224,47379292,47379326,47379383,47379411],"comment_text":"I feel like we're dancing on the razor's edge.
On the one hand: high inflation, tariffs, layoffs, unemployment, high interest rate, energy crisis. Tons of economic red flags flashing.
On the other hand: AI is showing signs of being the next industrial revolution, we're re-industrializing, onshoring/friendshoring, and have a clear lead on chips and space tech at a time when it matters the most.
It's absolutely insane that Claude Code can spit out a week's worth of business automation tasks in half a day. And do it at relatively high quality in low-defect rate languages like Rust.
Europe won't be able to catch that. They're too busy regulating ahead of the tech. They're going to be a decade behind if they keep it up.
If we cut the chip supply right as things take off, China might not either. In a runaway takeoff scenario, they replace all their factory workers with robots and quickly scale and cost optimize. If America is smart, we might be able to do that too.
Our growth could accelerate or crater. These are wild times. More exciting than the last 20.
America needs to start pumping out new energy projects. It needs to make friends with all of its former allies. And it needs to import PhD students.
And we do need factories and raw inputs. The robots will take over for humans within a decade. If we stick the landing, we could be the new China right here at home.
Edit: rate limited on replies, so updating my comment instead.
Edit 2: Europe supplies the EUV lithography, but intelligence manifests higher up the stack. If we're talking rate limits, lots of countries supply critical inputs. \nI'm saying that Europe hasn't made strides towards developing their own models and infra, and it doesn't look like it's even close to starting to attack this problem. I want it to.
Edit 3: What I'm saying is that these tailwinds might put America back into the position it was in post-WWII. Manufacturing, tech, and science powerhouse in all the places that matter. Peers a generation or two behind. That's literally where America was after the war, and it looks like we could be teeing up for a repeat if it all doesn't unravel first.
America needs to double down on investing in energy and factories now. It looks like it will pay off in a big way.
Edit 4:
> You think Europe won't be able to use Claude Code
I would be extremely geopolitically anxious to rely on another country's tech in a take off scenario. Those tokens might be diverted to US businesses and factories. Or the US might strong arm concessions out of Europe. Europe needs domestic capability for this now.
It's not just Europe and sovereign nations. Workers and labor capital will be effectively frozen out of participation if there aren't open source equivalents.
> This is an downright evil take on the current situation.
It's just reality. Multipolarity means we're going to see a lot more of this type of framing, because it's what's happening on the ground.","created_at":"2026-03-14T17:27:23Z","created_at_i":1773509243,"objectID":"47378944","parent_id":47378638,"story_id":47378638,"story_title":"What happens when US economic data becomes unreliable","story_url":"https://mitsloan.mit.edu/ideas-made-to-matter/what-happens-when-us-economic-data-becomes-unreliable","updated_at":"2026-03-16T17:35:26Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"thesssaism"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"We spent the last month modeling software budgets to figure out why velocity often feels so low even with senior teams. The short answer seems to be structural: about 80% of engineering time goes to non-differentiating infrastructure (auth, pipelines, CRUD) rather than unique business logic.
We call it the "Infrastructure Tax." We analyzed an anonymized $2.4M engineering spend, and honestly, the breakdown was depressing. Only about 20% of that budget went to features that actually differentiated the product. The rest? Auth flows, database provisioning, CI/CD pipelines, and writing the same user schema for the 50th time. Even with modern frameworks, it feels like we're constantly paying a toll just to get to the starting line.
To quantify this, we built a theoretical model (yeah, I know, models aren't reality, but bear with me). We took a standard startup scenario:
- 5 developers\n- $500k/year combined cost (conservative inputs)\n- Standard SaaS requirements
We mapped hours to "Setup/Boilerplate" vs. "Core Business Logic." Then we modeled an "AI-native" approach where the AI handles ~90% of that initial scaffolding\u2014not just writing functions, but wiring the stack together.
The math suggests a structural inversion. In the traditional model, you're looking at roughly an 80/20 split (Maintenance/Infra vs. Features). With heavy AI automation handling the boilerplate, that shifts toward 30/70.
For a 5-person team, the model implies saving roughly 900+ hours a year. But the real metric isn't hours saved; it's opportunity cost. If you ship 20 features instead of 12, the value difference is non-linear. It's the difference between finding product-market fit and running out of runway.
Obviously, this is where the skepticism kicks in (and rightfully so). There are massive caveats here:
- *Maintenance Burden:* Generative code is easy to create but can be a nightmare to debug. If the AI spits out spaghetti code, you haven't saved time; you've just deferred the pain to the maintenance phase.\n- *Architectural Lock-in:* Automated setup usually means rigid opinions. If your app doesn't fit the "standard SaaS" mold, fighting the generated abstraction might cost more than writing it from scratch.\n- *The "Black Box" Risk:* There's a real danger of junior devs generating infrastructure they don't fundamentally understand, leading to security holes that senior devs have to patch later.
We're using these models to think about internal resource allocation, but I'm curious if this matches your reality.
- Does the 80/20 split hold true for your stack, or have modern tools (Vercel/Supabase/etc.) already solved this for you?\n- Is "Opportunity Cost" a metric you actually track in engineering, or just something CFOs talk about?\n- For those using AI for scaffolding: are you paying for it later in debugging costs?
I suspect the "Infrastructure Tax" is the main reason software feels expensive, but maybe I'm underestimating the value of bespoke plumbing."},"title":{"matchLevel":"none","matchedWords":[],"value":"Tell HN: We modeled the cost of boilerplate (it's ~80% of the budget)"}},"_tags":["story","author_thesssaism","story_47260560","ask_hn"],"author":"thesssaism","children":[47260815],"created_at":"2026-03-05T11:51:33Z","created_at_i":1772711493,"num_comments":0,"objectID":"47260560","points":3,"story_id":47260560,"story_text":"We spent the last month modeling software budgets to figure out why velocity often feels so low even with senior teams. The short answer seems to be structural: about 80% of engineering time goes to non-differentiating infrastructure (auth, pipelines, CRUD) rather than unique business logic.
We call it the "Infrastructure Tax." We analyzed an anonymized $2.4M engineering spend, and honestly, the breakdown was depressing. Only about 20% of that budget went to features that actually differentiated the product. The rest? Auth flows, database provisioning, CI/CD pipelines, and writing the same user schema for the 50th time. Even with modern frameworks, it feels like we're constantly paying a toll just to get to the starting line.
To quantify this, we built a theoretical model (yeah, I know, models aren't reality, but bear with me). We took a standard startup scenario:
- 5 developers\n- $500k/year combined cost (conservative inputs)\n- Standard SaaS requirements
We mapped hours to "Setup/Boilerplate" vs. "Core Business Logic." Then we modeled an "AI-native" approach where the AI handles ~90% of that initial scaffolding\u2014not just writing functions, but wiring the stack together.
The math suggests a structural inversion. In the traditional model, you're looking at roughly an 80/20 split (Maintenance/Infra vs. Features). With heavy AI automation handling the boilerplate, that shifts toward 30/70.
For a 5-person team, the model implies saving roughly 900+ hours a year. But the real metric isn't hours saved; it's opportunity cost. If you ship 20 features instead of 12, the value difference is non-linear. It's the difference between finding product-market fit and running out of runway.
Obviously, this is where the skepticism kicks in (and rightfully so). There are massive caveats here:
- *Maintenance Burden:* Generative code is easy to create but can be a nightmare to debug. If the AI spits out spaghetti code, you haven't saved time; you've just deferred the pain to the maintenance phase.\n- *Architectural Lock-in:* Automated setup usually means rigid opinions. If your app doesn't fit the "standard SaaS" mold, fighting the generated abstraction might cost more than writing it from scratch.\n- *The "Black Box" Risk:* There's a real danger of junior devs generating infrastructure they don't fundamentally understand, leading to security holes that senior devs have to patch later.
We're using these models to think about internal resource allocation, but I'm curious if this matches your reality.
- Does the 80/20 split hold true for your stack, or have modern tools (Vercel/Supabase/etc.) already solved this for you?\n- Is "Opportunity Cost" a metric you actually track in engineering, or just something CFOs talk about?\n- For those using AI for scaffolding: are you paying for it later in debugging costs?
I suspect the "Infrastructure Tax" is the main reason software feels expensive, but maybe I'm underestimating the value of bespoke plumbing.","title":"Tell HN: We modeled the cost of boilerplate (it's ~80% of the budget)","updated_at":"2026-03-13T03:34:39Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"lifeisstillgood"},"comment_text":{"matchLevel":"none","matchedWords":[],"value":">> just aren't good for end to end agentic workflows.
This is (apparently) the conceit of SteveYegge / GasTown - no model can cope unassisted so chunk it up, run it and if it falls over remember the exact place and restart, merging it all in
But that\u2019s not my point.
I believe that software is a new form of literacy and just as all\nCompanies and societies are literate now, in the future (tm) companies will run exclusively on software - AI developed software and those who go all out will have the sort of advantages the Catholic Church had over .. guilds?
Anyhow, that\u2019s me being AI optimist. But writing the code is going to be a small part of that transition - almost everything to do with LLMs that is claimed amazing (Computer vision is something else) - almost everything people say we need an LLM is stuff you could have done three years ago but your internal politics just would not let you. Oh look we can see if our policies are being met (you could have written the policies in code and solved the whole problem)
Im struggling to get it out but - almost everything AI is proposed for is stuff a well run engineering firm coukd have taken on. A software literate firm could have done without AI is where firms are hoping AI will\nGet them
Imagine how far ahead real software literate firms will be - as long as they don\u2019t burn their runway in tokens
Which is why, the right play imo is still buy in-house as much as possible, engineer around the problems and explore the phase space at marginal\nCost.
Then and only then think frontier models."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Price per 1M tokens is meaningless"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://janilowski.pl/en/blog/2026/price-per-m-tokens/"}},"_tags":["comment","author_lifeisstillgood","story_48809542"],"author":"lifeisstillgood","comment_text":">> just aren't good for end to end agentic workflows.
This is (apparently) the conceit of SteveYegge / GasTown - no model can cope unassisted so chunk it up, run it and if it falls over remember the exact place and restart, merging it all in
But that\u2019s not my point.
I believe that software is a new form of literacy and just as all\nCompanies and societies are literate now, in the future (tm) companies will run exclusively on software - AI developed software and those who go all out will have the sort of advantages the Catholic Church had over .. guilds?
Anyhow, that\u2019s me being AI optimist. But writing the code is going to be a small part of that transition - almost everything to do with LLMs that is claimed amazing (Computer vision is something else) - almost everything people say we need an LLM is stuff you could have done three years ago but your internal politics just would not let you. Oh look we can see if our policies are being met (you could have written the policies in code and solved the whole problem)
Im struggling to get it out but - almost everything AI is proposed for is stuff a well run engineering firm coukd have taken on. A software literate firm could have done without AI is where firms are hoping AI will\nGet them
Imagine how far ahead real software literate firms will be - as long as they don\u2019t burn their runway in tokens
Which is why, the right play imo is still buy in-house as much as possible, engineer around the problems and explore the phase space at marginal\nCost.
Then and only then think frontier models.","created_at":"2026-07-06T20:55:33Z","created_at_i":1783371333,"objectID":"48810396","parent_id":48810020,"story_id":48809542,"story_title":"Price per 1M tokens is meaningless","story_url":"https://janilowski.pl/en/blog/2026/price-per-m-tokens/","updated_at":"2026-07-06T21:44:34Z"}],"hitsPerPage":20,"nbHits":22,"nbPages":2,"page":0,"params":"query=AI+agent+cost+runaway+loop&advancedSyntax=true&analyticsTags=backend","processingTimeMS":70,"processingTimingsMS":{"_request":{"roundTrip":14},"afterFetch":{"format":{"highlighting":4,"total":4},"merge":{"mergeLoop":{"prepareNextHit":16,"total":16},"total":22},"total":22},"fetch":{"query":14,"scanning":32,"total":47},"total":70},"query":"AI agent cost runaway loop","serverTimeMS":76}