And was pleased with what I was able to do. Thanks","title":null,"type":"comment","url":null},{"author":"isuckatcoding","children":[{"author":"honorable_coder","children":[],"created_at":"2025-07-13T16:50:16.000Z","created_at_i":1752425416,"id":44551693,"options":[],"parent_id":44551188,"points":null,"story_id":44546265,"text":"What\u2019s missing right now are our guides showing how well ArchGW integrates with existing frameworks and tools. But the core idea is simple: it offloads low-level responsibilities\u2014like routing, safety, and observability\u2014that frameworks like LangChain currently try to handle inside the app. That means less bloat and more clarity in your agent logic.
And importantly, some things just can\u2019t be done well in a framework. For example, enforcing global rate limits across LLMs isn\u2019t realistic when each agent instance holds its own local state. That kind of cross-cutting concern needs to live in infrastructure\u2014not in application code.","title":null,"type":"comment","url":null},{"author":"ethan_smith","children":[{"author":"honorable_coder","children":[],"created_at":"2025-07-14T13:22:39.000Z","created_at_i":1752499359,"id":44559908,"options":[],"parent_id":44558007,"points":null,"story_id":44546265,"text":"This ^","title":null,"type":"comment","url":null}],"created_at":"2025-07-14T09:26:09.000Z","created_at_i":1752485169,"id":44558007,"options":[],"parent_id":44551188,"points":null,"story_id":44546265,"text":"ArchGW complements langchain rather than replacing it - langchain handles agent orchestration/reasoning while ArchGW provides the infrastructure layer for prompt processing, guardrails and routing across your entire system.","title":null,"type":"comment","url":null}],"created_at":"2025-07-13T15:39:28.000Z","created_at_i":1752421168,"id":44551188,"options":[],"parent_id":44546265,"points":null,"story_id":44546265,"text":"I\u2019m still new to this ecosystem but is this something you\u2019d use together with langchain or does it replace some use cases there?","title":null,"type":"comment","url":null},{"author":"jufter","children":[{"author":"honorable_coder","children":[],"created_at":"2025-07-13T21:54:06.000Z","created_at_i":1752443646,"id":44554101,"options":[],"parent_id":44551466,"points":null,"story_id":44546265,"text":"We\u2019re using proxy-wasm and compiling to wasm32-wasip1, then mounting the .wasm binaries into Envoy as HTTP filters via envoy.filters.http.wasm. The line you're referring to:
vm_config:\n runtime: "envoy.wasm.runtime.v8"\n code:\n local:\n filename: "/etc/envoy/proxy-wasm-plugins/prompt_gateway.wasm"
\u2026is where the integration happens. There's no need to modify envoy.bootstrap.wasm; instead, Arch loads the WASM modules at runtime using standard Envoy config templating. The filters (prompt_gateway for ingress, and llm_gateway for egress sit in the request path and do things like prompt inspection, model routing, header rewrites, and telemetry collection.","title":null,"type":"comment","url":null}],"created_at":"2025-07-13T16:18:13.000Z","created_at_i":1752423493,"id":44551466,"options":[],"parent_id":44546265,"points":null,"story_id":44546265,"text":"Was going to ask how this integrates into Envoy but dug into the code it looks like proxywasm which must mean `envoy.bootstrap.wasm` ?","title":null,"type":"comment","url":null},{"author":"markanton","children":[{"author":"honorable_coder","children":[{"author":"chatmasta","children":[{"author":"honorable_coder","children":[],"created_at":"2025-07-14T17:43:00.000Z","created_at_i":1752514980,"id":44563023,"options":[],"parent_id":44562723,"points":null,"story_id":44546265,"text":"Its a core dependency for rate limiting, traffic shaping, fail over detection. Its cluster subsystem is super convenient for local LLM calls too. We'll write up a blog on the lessons because there were many. For example, for intelligent routing decisions we can't create an upstream connection to a cluster based on route paths or host - Envoy forces a more static binding. This doesn't work when you are making decisions about a prompt and have to inject more dynamic flow control.","title":null,"type":"comment","url":null}],"created_at":"2025-07-14T17:18:41.000Z","created_at_i":1752513521,"id":44562723,"options":[],"parent_id":44559901,"points":null,"story_id":44546265,"text":"fwiw, if I were evaluating these proxies against each other, I would be intrigued by the solution built by people from the Envoy team. Envoy is great software and I\u2019m sure there are many lessons you took from building it.
It looks like you\u2019re even building on Envoy as the foundation for the system which just makes it more compelling.","title":null,"type":"comment","url":null},{"author":"fosk","children":[{"author":"honorable_coder","children":[],"created_at":"2025-07-14T20:48:11.000Z","created_at_i":1752526091,"id":44565069,"options":[],"parent_id":44564521,"points":null,"story_id":44546265,"text":"MCP implementation is trivial - I agree. But A2A will require a mesh like structure. Meaning its not just about north/south traffic. It will be about east/west traffic as agents coordinate with each other. That communication and coordination among agents will need to be robust and that's where a sidecar proxy built on top of Envoy will offer certain properties in a first-class way that Kong can't easily support today.
This was the insight behind Envoy's initial design. Handle north/south and east/west traffic equally well as a universal data plane.","title":null,"type":"comment","url":null}],"created_at":"2025-07-14T19:53:04.000Z","created_at_i":1752522784,"id":44564521,"options":[],"parent_id":44559901,"points":null,"story_id":44546265,"text":"MCP is simply an API protocol, like GraphQL or gRPC.
And since everything is an API, Kong also supports MCP natively (among many other protocols, including all LLMs): https://konghq.com/blog/product-releases/securing-observing-...","title":null,"type":"comment","url":null}],"created_at":"2025-07-14T13:22:14.000Z","created_at_i":1752499334,"id":44559901,"options":[],"parent_id":44558678,"points":null,"story_id":44546265,"text":"There are a few critical differences. archgw is designed as a data plane for agents - handling and processing ingress and egress (prompt) traffic to/from agents. Unlike frameworks or libraries, it runs as a single process that includes edge functionality and task-specific LLMs, tightly integrated to reduce latency and complexity.
Second, it\u2019s about where the project is headed. Because archgw is built as a proxy server for agents, it\u2019s designed to support emerging low-level protocols like A2A and MCP in a consistent, unified way\u2014so developers can focus purely on high-level agent logic. This borrows from the same design decision that made Envoy successful for microservices: offload infrastructure concerns to a specialized layer, and keep application code clean. In our next big release, you will be able to run archgw as a sidecar proxy for improved orchestration and observability of agents. Something that other projects just won't be able to do.
Kong was designed for APIs. Envoy was built for microservices. Arch is built for agents.","title":null,"type":"comment","url":null}],"created_at":"2025-07-14T11:04:10.000Z","created_at_i":1752491050,"id":44558678,"options":[],"parent_id":44546265,"points":null,"story_id":44546265,"text":"Nice project but there are several dozens of \u201cAI/LLM gateways\u201d now.. all kind doing the same thing. Kong AI gateway [1] was maybe the first to attack the LLM traffic governance and is indeed far ahead in both features and adoption. Trying to understand the value add and differentiator here, since it\u2019s a problem kinda solved already.
[https://github.com/Kong/kong]","title":null,"type":"comment","url":null}],"created_at":"2025-07-12T23:55:39.000Z","created_at_i":1752364539,"id":44546265,"options":[],"parent_id":null,"points":118,"story_id":44546265,"text":"Hey HN!
This is Adil, Salman and Jose and and we\u2019re behind archgw [1]. An intelligent proxy server designed as an edge and AI gateway for agents - one that natively know how to handle prompts, not just network traffic. We\u2019ve made several sweeping changes so sharing the project again.
A bit of background on why we\u2019ve built this project. Building AI agent demos is easy, but to create something production-ready there is a lot of repeat low-level plumbing work that everyone is doing. You\u2019re applying guardrails to make sure unsafe or off-topic requests don\u2019t get through. You\u2019re clarifying vague input so agents don\u2019t make mistakes. You\u2019re routing prompts to the right expert agent based on context or task type. You\u2019re writing integration code to quickly and safely add support for new LLMs. And every time a new framework hits the market or is updated, you\u2019re validating or re-implementing that same logic\u2014again and again.
Putting all the low-level plumbing code in a framework gets messy to manage, harder to update and scale. Low-level work isn't business logic. That\u2019s why we built archgw - an intelligent proxy server that handles prompts during ingress and egress and offers several related capabilities from a single software service. It lives outside your app runtime, so you can keep your business logic clean and focus on what matters. Think of it like a service mesh, but for AI agents.
Prior to building archgw, the team spent time building Envoy [2] at Lyft, API Gateway at AWS, specialized NLP models at Microsoft Research and worked on safety at Meta. archgw was born out of the belief that rule-based, single-purpose tools that handle the work around resiliency, processing and routing prompts should move into a dedicated infrastructure layer for agents, but built on the battle-tested foundational of Envoy Proxy.
The intelligence in archgw comes from our fast Task-specific LLMs [3] that can handle things like agent routing and hand off, guardrails and preference-based intelligent LLM calling. Here are some additional details about the open source project. archgw is written in rust, and the request path has three main parts:
* Listener subsystem which handles downstream (ingress) and upstream (egress) request processing.\n* Prompt handler subsystem. This is where archgw makes decisions on the safety of the incoming request via its prompt_guard hooks and identifies where to forward the conversation to via its prompt_target primitive.\n* Model serving subsystem is the interface that hosts all the lightweight LLMs engineered in archgw and offers a framework for things like hallucination detection of our these models
We loved building this open source project, and our belief is that this infra primitive would help developers build faster, safer and more personalized agents without all the manual prompt engineering and systems integration work needed to get there. We hope to invite other developers to use and improve Arch. Please give it a shot and leave feedback here, or at our discord channel [4]\nAlso here is a quick demo of the project in action [5]. You can check out our public docs here at [6]. Our models are also available here [7].
[1] https://github.com/katanemo/archgw\n[2] https://www.envoyproxy.io/\n[3] https://huggingface.co/collections/katanemo/arch-function-66...\n[4] https://discord.com/channels/1292630766827737088/12926307682...\n[5] https://www.youtube.com/watch?v=I4Lbhr-NNXk\n[6] https://docs.archgw.com/\n[7] https://huggingface.co/katanemo","title":"Show HN: ArchGW \u2013 An intelligent edge and service proxy for agents","type":"story","url":"https://github.com/katanemo/archgw/"}