Today's agents either scrape (no consent, no structure) or the site builds a separate API (expensive, doesn't cover the long tail). The web's original protocols assumed someone is looking at a screen. That assumption is breaking.
We wrote a whitepaper mapping the full protocol landscape - Cloudflare's Pay Per Crawl and Web Bot Auth (RFC 9421), MCP, A2A, x402, llms.txt - and categorizing 5 distinct agent architectures (text-based, CUA/screenshot, DOM-based, API-calling, hybrid). Each needs different discovery, execution, and identity mechanisms. We think MCP, A2A, and execution protocols are complementary layers, not competitors. The paper draws parallels to TCP/HTTP design decisions.
Rover is our attempt at the execution layer. It's a DOM-native SDK the site owner installs. The Agent Task Protocol is one HTTP endpoint: POST /v1/tasks with { url, prompt }. Agents get back a task URL supporting JSON polling, SSE, or NDJSON. The site controls what agents can do and gets analytics on what they actually did. We're probably wrong about some of this -- would appreciate the feedback.
Paper: https://www.rtrvr.ai/blog/agent-web-protocol-stack \nCode: https://github.com/rtrvr-ai/rover (FSL-1.1-Apache-2.0)"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["agents"],"value":"Show HN: MCP is for tools. A2A is for agents. What's for websites?"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.rtrvr.ai/blog/agent-web-protocol-stack"}},"_tags":["story","author_quarkcarbon279","story_47736402","show_hn"],"author":"quarkcarbon279","created_at":"2026-04-12T05:30:17Z","created_at_i":1775971817,"num_comments":0,"objectID":"47736402","points":5,"story_id":47736402,"story_text":"HTTP lets agents fetch pages. Cloudflare's Markdown for Agents lets them fetch more efficiently. MCP (Anthropic) connects agents to developer-defined tools. A2A (Google) lets agents delegate to other agents. But there's a missing layer: how does an agent execute a multi-step task on a website -- add to cart, fill a form, complete a checkout - with the site owner's consent and visibility?
Today's agents either scrape (no consent, no structure) or the site builds a separate API (expensive, doesn't cover the long tail). The web's original protocols assumed someone is looking at a screen. That assumption is breaking.
We wrote a whitepaper mapping the full protocol landscape - Cloudflare's Pay Per Crawl and Web Bot Auth (RFC 9421), MCP, A2A, x402, llms.txt - and categorizing 5 distinct agent architectures (text-based, CUA/screenshot, DOM-based, API-calling, hybrid). Each needs different discovery, execution, and identity mechanisms. We think MCP, A2A, and execution protocols are complementary layers, not competitors. The paper draws parallels to TCP/HTTP design decisions.
Rover is our attempt at the execution layer. It's a DOM-native SDK the site owner installs. The Agent Task Protocol is one HTTP endpoint: POST /v1/tasks with { url, prompt }. Agents get back a task URL supporting JSON polling, SSE, or NDJSON. The site controls what agents can do and gets analytics on what they actually did. We're probably wrong about some of this -- would appreciate the feedback.
Paper: https://www.rtrvr.ai/blog/agent-web-protocol-stack \nCode: https://github.com/rtrvr-ai/rover (FSL-1.1-Apache-2.0)","title":"Show HN: MCP is for tools. A2A is for agents. What's for websites?","updated_at":"2026-04-12T16:01:15Z","url":"https://www.rtrvr.ai/blog/agent-web-protocol-stack"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"crawdog"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"I read Cloudflare's "Markdown for Agents" article and thought this would be really powerful if it could support internal websites, JSON content and dynamic Javascript sites.
I built this on my 35-minute BART ride to SF using my phone as the IDE (and a couple of hours of feature creep/testing). It's a combination MCP server + HTTP proxy that converts web content to markdown on-the-fly.
Key features:
* HTTP/HTTPS proxy with automatic HTML to markdown conversion\n* Token counting and caching to stay efficient\n* Content negotiation (only converts when Accept: text/markdown)\n* MCP server\n* JavaScript rendering for dynamic sites
Let me know what you think!"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["markdown"],"value":"Show HN: Markdown in the Middle \u2013 proxy to convert HTML to Markdown"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/rickcrawford/markdowninthemiddle"}},"_tags":["story","author_crawdog","story_47134859","show_hn"],"author":"crawdog","created_at":"2026-02-24T09:24:34Z","created_at_i":1771925074,"num_comments":0,"objectID":"47134859","points":2,"story_id":47134859,"story_text":"I read Cloudflare's "Markdown for Agents" article and thought this would be really powerful if it could support internal websites, JSON content and dynamic Javascript sites.
I built this on my 35-minute BART ride to SF using my phone as the IDE (and a couple of hours of feature creep/testing). It's a combination MCP server + HTTP proxy that converts web content to markdown on-the-fly.
Key features:
* HTTP/HTTPS proxy with automatic HTML to markdown conversion\n* Token counting and caching to stay efficient\n* Content negotiation (only converts when Accept: text/markdown)\n* MCP server\n* JavaScript rendering for dynamic sites
Let me know what you think!","title":"Show HN: Markdown in the Middle \u2013 proxy to convert HTML to Markdown","updated_at":"2026-03-05T23:36:12Z","url":"https://github.com/rickcrawford/markdowninthemiddle"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"frumu"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"I ran into Cloudflare\u2019s Markdown for Agents and thought it was exactly what I needed for LLM web research. Then I realized it only helps when a site is on Cloudflare and has it enabled, so it doesn\u2019t solve \u201copen web\u201d extraction.
I built a simple HTML\u2192Markdown pipeline in Rust that works on any public URL (strip scripts/styles/boilerplate, preserve structure + links). On a 100-URL set it reduced input size by ~70\u201380% (often close to 80%).
Benchmark on the same 100 URLs:
Rust server mode: p50 ~0.4s, p95 ~1.3s, memory ~100MB stable
Node baseline (JSDOM + Turndown): p50 ~1.2s, p95 ~50s, memory grew into hundreds of MB to GBs
Scripts + methodology are in the repo: <link>
Curious what others use for boilerplate removal and how you keep p95 tails under control when parsing nasty pages."},"story_title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"From Cloudflare's Markdown for Agents to a Universal HTML\u2192Markdown Extractor"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/frumu-ai/tandem"}},"_tags":["comment","author_frumu","story_47113352"],"author":"frumu","comment_text":"I ran into Cloudflare\u2019s Markdown for Agents and thought it was exactly what I needed for LLM web research. Then I realized it only helps when a site is on Cloudflare and has it enabled, so it doesn\u2019t solve \u201copen web\u201d extraction.
I built a simple HTML\u2192Markdown pipeline in Rust that works on any public URL (strip scripts/styles/boilerplate, preserve structure + links). On a 100-URL set it reduced input size by ~70\u201380% (often close to 80%).
Benchmark on the same 100 URLs:
Rust server mode: p50 ~0.4s, p95 ~1.3s, memory ~100MB stable
Node baseline (JSDOM + Turndown): p50 ~1.2s, p95 ~50s, memory grew into hundreds of MB to GBs
Scripts + methodology are in the repo: <link>
Curious what others use for boilerplate removal and how you keep p95 tails under control when parsing nasty pages.","created_at":"2026-02-22T18:23:28Z","created_at_i":1771784608,"objectID":"47113353","parent_id":47113352,"story_id":47113352,"story_title":"From Cloudflare's Markdown for Agents to a Universal HTML\u2192Markdown Extractor","story_url":"https://github.com/frumu-ai/tandem","updated_at":"2026-03-05T23:35:01Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"elbeyoglu"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"The pitch: prepend markdown.new/ to any URL, get clean Markdown back. One line. Works anywhere you'd otherwise write a scraper.
I built the first version of markdown.new about a year ago. The problem was obvious: AI pipelines were choking on raw HTML, and every team building RAG systems was writing the same boilerplate \u2014 Puppeteer for rendering, BeautifulSoup for stripping, custom regex for cleanup, separate handlers for PDFs. I wanted a single URL prepend to collapse all of that.\nIt worked, people used it, but it didn't exactly take the internet by storm.
Then on February 12, 2026, Cloudflare announced Markdown for Agents \u2014 a native feature that lets AI agents request Markdown directly from any Cloudflare-enabled site using a simple Accept: text/markdown header. The network converts HTML to Markdown at the edge, on the fly, and returns it with token count estimates in the response headers. No extra tooling needed on the client side.
It was exactly the infrastructure-level solution the problem deserved. I immediately integrated it as the primary tier in markdown.new's conversion pipeline. Two days later I shipped the updated version. By end of day the announcement tweet had nearly a million views.
Sometimes you build the right thing a year too early. Then the ecosystem catches up, and suddenly it all clicks."},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["markdown","agents"],"value":"I built markdown.new (Markdown for Agents) and it went a little viral on X"},"story_url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["markdown"],"value":"https://markdown.new"}},"_tags":["comment","author_elbeyoglu","story_47085039"],"author":"elbeyoglu","comment_text":"The pitch: prepend markdown.new/ to any URL, get clean Markdown back. One line. Works anywhere you'd otherwise write a scraper.
I built the first version of markdown.new about a year ago. The problem was obvious: AI pipelines were choking on raw HTML, and every team building RAG systems was writing the same boilerplate \u2014 Puppeteer for rendering, BeautifulSoup for stripping, custom regex for cleanup, separate handlers for PDFs. I wanted a single URL prepend to collapse all of that.\nIt worked, people used it, but it didn't exactly take the internet by storm.
Then on February 12, 2026, Cloudflare announced Markdown for Agents \u2014 a native feature that lets AI agents request Markdown directly from any Cloudflare-enabled site using a simple Accept: text/markdown header. The network converts HTML to Markdown at the edge, on the fly, and returns it with token count estimates in the response headers. No extra tooling needed on the client side.
It was exactly the infrastructure-level solution the problem deserved. I immediately integrated it as the primary tier in markdown.new's conversion pipeline. Two days later I shipped the updated version. By end of day the announcement tweet had nearly a million views.
Sometimes you build the right thing a year too early. Then the ecosystem catches up, and suddenly it all clicks.","created_at":"2026-02-20T07:57:31Z","created_at_i":1771574251,"objectID":"47085040","parent_id":47085039,"story_id":47085039,"story_title":"I built markdown.new (Markdown for Agents) and it went a little viral on X","story_url":"https://markdown.new","updated_at":"2026-03-05T23:36:16Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"juanisidoro"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"Hi HN, I built MAKO after watching Cloudflare's Markdown for Agents launch last week. They validated the problem \u2014 AI agents waste tokens on HTML bloat. But their approach is automatic conversion with no structure.
MAKO provides what's missing:
The Protocol:\nHTTP content negotiation: agent sends `Accept: text/mako+markdown`, server responds with structured content. Standard HTTP, no new endpoints.
What Makes It Different:\n- YAML frontmatter with metadata (type, entity, actions, semantic links)\n- Optimized markdown body (not auto-converted \u2014 semantically curated)\n- Embeddings in headers (CEF format, ~470 bytes) for pre-download relevance filtering\n- Per-page granularity (vs llms.txt which is per-site)
The Numbers:\n- E-commerce product: 47,000 HTML tokens \u2192 320 MAKO tokens (93% reduction)\n- Blog article: 35,000 \u2192 670 tokens (98%)\n- Landing page: 110,000 \u2192 640 tokens (99%)
What's Included (Apache 2.0):\n- Spec: https://github.com/juanisidoro/mako-spec\n- JS SDK: `npm install @mako-spec/js` (parser, generator, validator, middleware)\n- CLI: `npm install @mako-spec/cli`\n- WordPress plugin (WooCommerce support)\n- Free scoring tool: https://makospec.vercel.app/en/score
Complements Existing Standards:\n- llms.txt (site-level guidance) \u2192 MAKO (page-level content)\n- Cloudflare MD (auto-converted) \u2192 MAKO (semantically optimized)\n- WebMCP (actions only) \u2192 MAKO (content + actions)\n- Schema.org (for search engines) \u2192 MAKO (for AI agents)
Feedback welcome on the spec, scoring criteria, or anything else. Happy to answer questions."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: MAKO \u2013 Open protocol for LLM-optimized web content (93% fewer tokens)"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://makospec.vercel.app/en"}},"_tags":["comment","author_juanisidoro","story_47049011"],"author":"juanisidoro","comment_text":"Hi HN, I built MAKO after watching Cloudflare's Markdown for Agents launch last week. They validated the problem \u2014 AI agents waste tokens on HTML bloat. But their approach is automatic conversion with no structure.
MAKO provides what's missing:
The Protocol:\nHTTP content negotiation: agent sends `Accept: text/mako+markdown`, server responds with structured content. Standard HTTP, no new endpoints.
What Makes It Different:\n- YAML frontmatter with metadata (type, entity, actions, semantic links)\n- Optimized markdown body (not auto-converted \u2014 semantically curated)\n- Embeddings in headers (CEF format, ~470 bytes) for pre-download relevance filtering\n- Per-page granularity (vs llms.txt which is per-site)
The Numbers:\n- E-commerce product: 47,000 HTML tokens \u2192 320 MAKO tokens (93% reduction)\n- Blog article: 35,000 \u2192 670 tokens (98%)\n- Landing page: 110,000 \u2192 640 tokens (99%)
What's Included (Apache 2.0):\n- Spec: https://github.com/juanisidoro/mako-spec\n- JS SDK: `npm install @mako-spec/js` (parser, generator, validator, middleware)\n- CLI: `npm install @mako-spec/cli`\n- WordPress plugin (WooCommerce support)\n- Free scoring tool: https://makospec.vercel.app/en/score
Complements Existing Standards:\n- llms.txt (site-level guidance) \u2192 MAKO (page-level content)\n- Cloudflare MD (auto-converted) \u2192 MAKO (semantically optimized)\n- WebMCP (actions only) \u2192 MAKO (content + actions)\n- Schema.org (for search engines) \u2192 MAKO (for AI agents)
Feedback welcome on the spec, scoring criteria, or anything else. Happy to answer questions.","created_at":"2026-02-17T16:06:31Z","created_at_i":1771344391,"objectID":"47049025","parent_id":47049011,"story_id":47049011,"story_title":"Show HN: MAKO \u2013 Open protocol for LLM-optimized web content (93% fewer tokens)","story_url":"https://makospec.vercel.app/en","updated_at":"2026-03-05T23:34:29Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"vaibhavlodha98"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"Hey HN! I'm the creator of Klovr.
I built this to solve a problem I had when building AI agents: HTML wastes 60-95% of tokens, and Cloudflare's new "Markdown for Agents" only works on\n ~5% of the web (opt-in only).\n\n THE PROBLEM:\n I tested 100 popular websites with Cloudflare's Accept: text/markdown header. Only 3 actually served markdown. The rest? Still HTML. Turns out their\n markdown feature requires website owners to opt-in, which most won't do for years (if ever).\n\n MY SOLUTION:\n Klovr converts any webpage to markdown on-demand:\n - Same Accept headers as Cloudflare (100% compatible)\n - Works on 100% of sites (no opt-in needed)\n - Redis caching with 7-day TTL (10-100x speedup on repeated URLs)\n - Playwright for dynamic content (better anti-detection than Puppeteer)\n - Content-Signal headers for AI compliance\n\n TECH STACK:\n - Next.js 15 (App Router) + Vercel\n - Playwright for browser automation\n - Redis (via ioredis) for caching\n - Drizzle ORM + Neon PostgreSQL\n - Readability.js + Turndown for conversion\n\n FREE TIER: 10,000 conversions/month (no credit card)\n\n WHAT I LEARNED:\n 1. Puppeteer-extra doesn't work on Vercel (ESM/CommonJS conflicts)\n 2. Playwright has better anti-detection out of the box\n 3. Redis caching is critical - first request is 2000ms, cached is 50ms\n 4. Most sites still don't support Cloudflare's markdown (hence the need for universal conversion)\n\n CURRENT LIMITATIONS:\n - Payment processing is in development (everyone on free tier for now)\n - Dynamic content (Playwright) temporarily disabled for launch (re-enabling next week)\n - IP-based blocking (Reddit, LinkedIn) still happens - no way around datacenter IPs\n\n I'd love feedback on:\n - Architecture choices (should I use a different caching strategy?)\n - The positioning (am I framing the Cloudflare comparison correctly?)\n - What features would make this more useful for your AI agents?\n\n GitHub isn't public yet, but happy to share code snippets for specific parts (stealth script, caching layer, etc.). I built this to solve a problem I had when building AI agents: HTML wastes 60-95% of tokens, and Cloudflare's new "Markdown for Agents" only works on\n ~5% of the web (opt-in only).\n\n THE PROBLEM:\n I tested 100 popular websites with Cloudflare's Accept: text/markdown header. Only 3 actually served markdown. The rest? Still HTML. Turns out their\n markdown feature requires website owners to opt-in, which most won't do for years (if ever).\n\n MY SOLUTION:\n Klovr converts any webpage to markdown on-demand:\n - Same Accept headers as Cloudflare (100% compatible)\n - Works on 100% of sites (no opt-in needed)\n - Redis caching with 7-day TTL (10-100x speedup on repeated URLs)\n - Playwright for dynamic content (better anti-detection than Puppeteer)\n - Content-Signal headers for AI compliance\n\n TECH STACK:\n - Next.js 15 (App Router) + Vercel\n - Playwright for browser automation\n - Redis (via ioredis) for caching\n - Drizzle ORM + Neon PostgreSQL\n - Readability.js + Turndown for conversion\n\n FREE TIER: 10,000 conversions/month (no credit card)\n\n WHAT I LEARNED:\n 1. Puppeteer-extra doesn't work on Vercel (ESM/CommonJS conflicts)\n 2. Playwright has better anti-detection out of the box\n 3. Redis caching is critical - first request is 2000ms, cached is 50ms\n 4. Most sites still don't support Cloudflare's markdown (hence the need for universal conversion)\n\n CURRENT LIMITATIONS:\n - Payment processing is in development (everyone on free tier for now)\n - Dynamic content (Playwright) temporarily disabled for launch (re-enabling next week)\n - IP-based blocking (Reddit, LinkedIn) still happens - no way around datacenter IPs\n\n I'd love feedback on:\n - Architecture choices (should I use a different caching strategy?)\n - The positioning (am I framing the Cloudflare comparison correctly?)\n - What features would make this more useful for your AI agents?\n\n GitHub isn't public yet, but happy to share code snippets for specific parts (stealth script, caching layer, etc.).Introducing mkdnsite ("markdown site") - an open source Markdown-native web server that serves HTML to humans and raw Markdown to agents. No build step required. Runs on Bun/Node/Deno, as an OS-specific standalone executable, or as a Docker container. Possibly the easiest way to go from Markdown files to functional website in the new agentic era.
Features:
- Runtime-only, zero build
- Content negotiation means HTML for browsers and Markdown for agents
- Supports GitHub-Flavored Markdown rendering
- Mermaid diagrams, KaTeX math, embedded Chart.js charts, syntax highlighting all included
- Full-text search for humans, MCP tools for agents
- Customizable UI theming with auto-support for light/dark modes
- Pull Markdown files directly from a GitHub repo
See official docs at https://mkdn.site
# Why?
Back in February, I saw Cloudflare's announcement of "Markdown for Agents" (https://blog.cloudflare.com/markdown-for-agents/). At the time, I thought "so I'm writing my API docs or blog in Markdown and converting it to HTML for a website, only to have Cloudflare turn it back into Markdown for AI/agent consumption". This seemed odd to me.
I'm a Node.js developer but had recently been building projects on Bun because of the "batteries included" features, like cross-compilation of standalone executables (similar to Go), that Node.js lacked natively (yes, I'm aware of Node SEA, but it's messy/complicated and `bun build --compile` is not).
Then, when I found `Bun.markdown`, something clicked for me - building a web server that converts Markdown to HTML at runtime should be super easy. And agents actually want Markdown, so why not combine the two ideas?
Humans like writing Markdown (well, at least, I do) and agents like reading Markdown (less verbose, easier to grok, fewer tokens). Add to this the fact that we can now use AI to write software, and my side project was born.
Is Markdown-to-HTML a new concept? Absolutely not. It's pretty old and well-established. But what I think is new is the ability to do everything at runtime (no build step required) and the built-in support for AI agents. mkdnsite has content negotiation, automated llms.txt, an MCP server, and support for agent headers.
# How?
I worked with Claude to refine the idea and come up with basic requirements/specs and then had Claude build me a scaffolded project. I started the project on March 7.
The following Friday, I configured my first set of OpenClaw agents on my personal machine and set them up to use Slack. From that point on, I spent most evenings and every weekend building mkdnsite and a hosted service (at https://mkdn.io) by logging ideas as issues in GitHub and talking with my "team lead" agent on Slack to pick up the work and implement features.
mkdnsite v1.0.0 was released on March 16. The current version is v1.4.1 released March 28. Almost every line of code was written by AI, either via an autonomous OpenClaw agent or via individual Claude Code sessions.
# So what?
Just looking for some honest feedback. Is this useful? Is it dumb? Is there another tool that offers the same combination of features (I looked and couldn't find one)?
I am not downplaying SSGs at all. I quite like Astro. And I love GitHub Pages. I just think there's room for an easier/simpler solution.
Please try it out and let me know what you think. Thanks."},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["markdown","agents"],"value":"Show HN: Mkdnsite \u2013 Markdown-native web server for humans (HTML) and agents (md)"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/mkdnsite/mkdnsite"}},"_tags":["story","author_nexdrew","story_47606946","show_hn"],"author":"nexdrew","created_at":"2026-04-01T21:47:32Z","created_at_i":1775080052,"num_comments":0,"objectID":"47606946","points":6,"story_id":47606946,"story_text":"# What?
Introducing mkdnsite ("markdown site") - an open source Markdown-native web server that serves HTML to humans and raw Markdown to agents. No build step required. Runs on Bun/Node/Deno, as an OS-specific standalone executable, or as a Docker container. Possibly the easiest way to go from Markdown files to functional website in the new agentic era.
Features:
- Runtime-only, zero build
- Content negotiation means HTML for browsers and Markdown for agents
- Supports GitHub-Flavored Markdown rendering
- Mermaid diagrams, KaTeX math, embedded Chart.js charts, syntax highlighting all included
- Full-text search for humans, MCP tools for agents
- Customizable UI theming with auto-support for light/dark modes
- Pull Markdown files directly from a GitHub repo
See official docs at https://mkdn.site
# Why?
Back in February, I saw Cloudflare's announcement of "Markdown for Agents" (https://blog.cloudflare.com/markdown-for-agents/). At the time, I thought "so I'm writing my API docs or blog in Markdown and converting it to HTML for a website, only to have Cloudflare turn it back into Markdown for AI/agent consumption". This seemed odd to me.
I'm a Node.js developer but had recently been building projects on Bun because of the "batteries included" features, like cross-compilation of standalone executables (similar to Go), that Node.js lacked natively (yes, I'm aware of Node SEA, but it's messy/complicated and `bun build --compile` is not).
Then, when I found `Bun.markdown`, something clicked for me - building a web server that converts Markdown to HTML at runtime should be super easy. And agents actually want Markdown, so why not combine the two ideas?
Humans like writing Markdown (well, at least, I do) and agents like reading Markdown (less verbose, easier to grok, fewer tokens). Add to this the fact that we can now use AI to write software, and my side project was born.
Is Markdown-to-HTML a new concept? Absolutely not. It's pretty old and well-established. But what I think is new is the ability to do everything at runtime (no build step required) and the built-in support for AI agents. mkdnsite has content negotiation, automated llms.txt, an MCP server, and support for agent headers.
# How?
I worked with Claude to refine the idea and come up with basic requirements/specs and then had Claude build me a scaffolded project. I started the project on March 7.
The following Friday, I configured my first set of OpenClaw agents on my personal machine and set them up to use Slack. From that point on, I spent most evenings and every weekend building mkdnsite and a hosted service (at https://mkdn.io) by logging ideas as issues in GitHub and talking with my "team lead" agent on Slack to pick up the work and implement features.
mkdnsite v1.0.0 was released on March 16. The current version is v1.4.1 released March 28. Almost every line of code was written by AI, either via an autonomous OpenClaw agent or via individual Claude Code sessions.
# So what?
Just looking for some honest feedback. Is this useful? Is it dumb? Is there another tool that offers the same combination of features (I looked and couldn't find one)?
I am not downplaying SSGs at all. I quite like Astro. And I love GitHub Pages. I just think there's room for an easier/simpler solution.
Please try it out and let me know what you think. Thanks.","title":"Show HN: Mkdnsite \u2013 Markdown-native web server for humans (HTML) and agents (md)","updated_at":"2026-04-05T18:48:46Z","url":"https://github.com/mkdnsite/mkdnsite"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"selcuka"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"Not the same thing, but they have something close (it's not on-by-default, yet) [1]:
> Cloudflare's network now supports real-time content conversion at the source, for enabled zones using content negotiation headers. Now when AI systems request pages from any website that uses Cloudflare and has Markdown for Agents enabled, they can express the preference for text/markdown in the request. Our network will automatically and efficiently convert the HTML to markdown, when possible, on the fly.
[1] https://blog.cloudflare.com/markdown-for-agents/"},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["cloudflare"],"value":"Cloudflare crawl endpoint"},"story_url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["cloudflare"],"value":"https://developers.cloudflare.com/changelog/post/2026-03-10-br-crawl-endpoint/"}},"_tags":["comment","author_selcuka","story_47329557"],"author":"selcuka","children":[47334595,47347451],"comment_text":"Not the same thing, but they have something close (it's not on-by-default, yet) [1]:
> Cloudflare's network now supports real-time content conversion at the source, for enabled zones using content negotiation headers. Now when AI systems request pages from any website that uses Cloudflare and has Markdown for Agents enabled, they can express the preference for text/markdown in the request. Our network will automatically and efficiently convert the HTML to markdown, when possible, on the fly.
[1] https://blog.cloudflare.com/markdown-for-agents/","created_at":"2026-03-11T00:24:21Z","created_at_i":1773188661,"objectID":"47330469","parent_id":47329920,"story_id":47329557,"story_title":"Cloudflare crawl endpoint","story_url":"https://developers.cloudflare.com/changelog/post/2026-03-10-br-crawl-endpoint/","updated_at":"2026-03-27T17:35:24Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"lgvld"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"We kind of do already: https://blog.cloudflare.com/markdown-for-agents/"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"I'm helping my dog vibe code games"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.calebleak.com/posts/dog-game/"}},"_tags":["comment","author_lgvld","story_47139675"],"author":"lgvld","comment_text":"We kind of do already: https://blog.cloudflare.com/markdown-for-agents/","created_at":"2026-02-25T11:19:48Z","created_at_i":1772018388,"objectID":"47150104","parent_id":47149937,"story_id":47139675,"story_title":"I'm helping my dog vibe code games","story_url":"https://www.calebleak.com/posts/dog-game/","updated_at":"2026-03-19T14:17:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jauntywundrkind"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"Made before cloudflare launched text/markdown, nice! If the timestamps are to be believed!
https://blog.cloudflare.com/markdown-for-agents/ https://news.ycombinator.com/item?id=46997526"},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["markdown"],"value":"Reader \u2013 web scraping that outputs clean Markdown for LLMs"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/vakra-dev/reader"}},"_tags":["comment","author_jauntywundrkind","story_47104954"],"author":"jauntywundrkind","comment_text":"Made before cloudflare launched text/markdown, nice! If the timestamps are to be believed!
https://blog.cloudflare.com/markdown-for-agents/ https://news.ycombinator.com/item?id=46997526","created_at":"2026-02-22T07:36:54Z","created_at_i":1771745814,"objectID":"47109090","parent_id":47104954,"story_id":47104954,"story_title":"Reader \u2013 web scraping that outputs clean Markdown for LLMs","story_url":"https://github.com/vakra-dev/reader","updated_at":"2026-03-05T23:34:45Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Penligentai"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"\u6b63\u6587\u6587\u6848\uff08HN \u53e3\u543b\uff0c\u77ed\u6bb5\u843d\uff0c\u4e0d\u5c2c\u5439\uff09
I\u2019ve been noticing a pattern lately: people are using Gemini (and other frontier models) for vulnerability research\u2014reading code, summarizing attack surfaces, generating exploit hypotheses, and iterating quickly.
One reason AI hasn\u2019t been applied everywhere by gray/black hat actors (at least indiscriminately) is that high-volume workflows still burn a lot of tokens. That cost adds up fast when you\u2019re doing broad recon, mass triage, or running \u201cagentic\u201d loops over huge corpuses.
Cloudflare just shipped \u201cMarkdown for Agents,\u201d which serves an agent-friendly Markdown version of pages and claims ~80% token reduction vs HTML for the same content. Combine that with gateway-side caching / edge optimizations and it feels like a meaningful change in the economics of large-scale AI-assisted scanning/triage.
Curious what people here think: does lowering token consumption materially shift offensive capability, or do other constraints dominate (rate limits, data quality, sandboxing, operational security, detection)? If you\u2019re doing vuln research with LLMs today, what\u2019s actually your limiting factor?"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"[dead]"}},"_tags":["comment","author_Penligentai","story_47013194"],"author":"Penligentai","comment_text":"\u6b63\u6587\u6587\u6848\uff08HN \u53e3\u543b\uff0c\u77ed\u6bb5\u843d\uff0c\u4e0d\u5c2c\u5439\uff09
I\u2019ve been noticing a pattern lately: people are using Gemini (and other frontier models) for vulnerability research\u2014reading code, summarizing attack surfaces, generating exploit hypotheses, and iterating quickly.
One reason AI hasn\u2019t been applied everywhere by gray/black hat actors (at least indiscriminately) is that high-volume workflows still burn a lot of tokens. That cost adds up fast when you\u2019re doing broad recon, mass triage, or running \u201cagentic\u201d loops over huge corpuses.
Cloudflare just shipped \u201cMarkdown for Agents,\u201d which serves an agent-friendly Markdown version of pages and claims ~80% token reduction vs HTML for the same content. Combine that with gateway-side caching / edge optimizations and it feels like a meaningful change in the economics of large-scale AI-assisted scanning/triage.
Curious what people here think: does lowering token consumption materially shift offensive capability, or do other constraints dominate (rate limits, data quality, sandboxing, operational security, detection)? If you\u2019re doing vuln research with LLMs today, what\u2019s actually your limiting factor?","created_at":"2026-02-14T09:51:20Z","created_at_i":1771062680,"objectID":"47013195","parent_id":47013194,"story_id":47013194,"story_title":"[dead]","updated_at":"2026-03-05T23:32:33Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"hval"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"A good validation for accept.md
https://blog.cloudflare.com/markdown-for-agents/
Markdown for agents by Cloudflare"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Accept-md \u2013 One command to make Next.js sites LLM-scraping friendly"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.accept.md/"}},"_tags":["comment","author_hval","story_46903806"],"author":"hval","comment_text":"A good validation for accept.md
https://blog.cloudflare.com/markdown-for-agents/
Markdown for agents by Cloudflare","created_at":"2026-02-13T15:18:11Z","created_at_i":1770995891,"objectID":"47003656","parent_id":46903806,"story_id":46903806,"story_title":"Show HN: Accept-md \u2013 One command to make Next.js sites LLM-scraping friendly","story_url":"https://www.accept.md/","updated_at":"2026-03-05T23:31:49Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"thestackfox"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"Cloudflare adds real-time Markdown rendering for AI agents"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"https://blog.cloudflare.com/markdown-for-agents/"}},"_tags":["story","author_thestackfox","story_46997526"],"author":"thestackfox","children":[46997527,46997645,46998808,46998945,46999052,46999518,46999811,46999949,47000087,47000400,47001192,47002842,47010564,47019511],"created_at":"2026-02-13T00:55:18Z","created_at_i":1770944118,"num_comments":22,"objectID":"46997526","points":48,"story_id":46997526,"title":"Cloudflare adds real-time Markdown rendering for AI agents","updated_at":"2026-04-20T18:22:31Z","url":"https://blog.cloudflare.com/markdown-for-agents/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"emot"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["markdown","agents"],"value":"We auto-convert HTML to Markdown for AI agents"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"https://blog.cloudflare.com/markdown-for-agents/"}},"_tags":["story","author_emot","story_46991030"],"author":"emot","children":[46991139,46991739],"created_at":"2026-02-12T16:43:59Z","created_at_i":1770914639,"num_comments":3,"objectID":"46991030","points":12,"story_id":46991030,"title":"We auto-convert HTML to Markdown for AI agents","updated_at":"2026-03-05T23:35:35Z","url":"https://blog.cloudflare.com/markdown-for-agents/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Bender"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["cloudflare","agents"],"value":"Cloudflare turns websites into faster food for AI agents"},"url":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["cloudflare","markdown"],"value":"https://www.theregister.com/2026/02/13/cloudflare_markdown_for_ai_crawlers/"}},"_tags":["story","author_Bender","story_47024877"],"author":"Bender","created_at":"2026-02-15T16:15:32Z","created_at_i":1771172132,"num_comments":0,"objectID":"47024877","points":3,"story_id":47024877,"title":"Cloudflare turns websites into faster food for AI agents","updated_at":"2026-03-05T23:33:08Z","url":"https://www.theregister.com/2026/02/13/cloudflare_markdown_for_ai_crawlers/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"janwilmake"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"I just built XYMake (https://xymake.com), a tool that lets you convert any X (Twitter) thread into clean markdown, making your conversations accessible for LLMs, MCPs, or any API.
## What it does:\n- Transforms any X thread URL into markdown by simply changing "x.com" to "xymake.com" in the URL\n- OAuth2 login to "free your data" and make your threads available\n- Auto-generates OG images with token counts and participant info for easy sharing\n- Serves different content types based on whether the request is from a crawler, browser, or agent
## Why I built it:\nI believe people should have the right to own and use their own data. While X/Twitter uses our content to train Grok, we should be able to leverage our own conversations for similar purposes.
I built this as a proof of concept in one day (what started as a 30-minute experiment turned into a 10-hour flow state). It's built entirely on Cloudflare Workers and uses some interesting techniques to serve different content types to different consumers.
## Technical highlights:\n- Request identification to serve HTML+OG images to crawlers while providing raw markdown to agents\n- Preloading OG image generation using ctx.waitUntil for near-instant loading when shared\n- Optimized OG image rendering across platforms using workers-og
Try it out with any X thread - just replace "x.com" with "xymake.com"!
Example: https://xymake.com/janwilmake/status/1904080559422288062
Feedback welcome! This is just the beginning of what's possible when we reclaim our conversational data."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: XYMake \u2013 Turn Your Posts into LLM-Ready Data"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://xymake.com"}},"_tags":["story","author_janwilmake","story_43464320","show_hn"],"author":"janwilmake","children":[43467668],"created_at":"2025-03-24T19:02:15Z","created_at_i":1742842935,"num_comments":1,"objectID":"43464320","points":8,"story_id":43464320,"story_text":"I just built XYMake (https://xymake.com), a tool that lets you convert any X (Twitter) thread into clean markdown, making your conversations accessible for LLMs, MCPs, or any API.
## What it does:\n- Transforms any X thread URL into markdown by simply changing "x.com" to "xymake.com" in the URL\n- OAuth2 login to "free your data" and make your threads available\n- Auto-generates OG images with token counts and participant info for easy sharing\n- Serves different content types based on whether the request is from a crawler, browser, or agent
## Why I built it:\nI believe people should have the right to own and use their own data. While X/Twitter uses our content to train Grok, we should be able to leverage our own conversations for similar purposes.
I built this as a proof of concept in one day (what started as a 30-minute experiment turned into a 10-hour flow state). It's built entirely on Cloudflare Workers and uses some interesting techniques to serve different content types to different consumers.
## Technical highlights:\n- Request identification to serve HTML+OG images to crawlers while providing raw markdown to agents\n- Preloading OG image generation using ctx.waitUntil for near-instant loading when shared\n- Optimized OG image rendering across platforms using workers-og
Try it out with any X thread - just replace "x.com" with "xymake.com"!
Example: https://xymake.com/janwilmake/status/1904080559422288062
Feedback welcome! This is just the beginning of what's possible when we reclaim our conversational data.","title":"Show HN: XYMake \u2013 Turn Your Posts into LLM-Ready Data","updated_at":"2025-03-25T22:35:48Z","url":"https://xymake.com"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"janwilmake"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["cloudflare","markdown","agents"],"value":"The web\u2019s stuck serving one format for all\u2014HTML for humans, but what about bots, APIs, or social previews? It\u2019s a legacy mess.
Here\u2019s the thing: one page should adapt\u2014HTML for crawlers, JSON for APIs, Markdown for agents, and PNG for social. Same data, smartly served.
I built a Cloudflare Worker to ditch legacy bloat. Vanilla JS/HTML/CSS, no frameworks, just a shared layer for all clients. First principles.
Meet my User-Agent Router: a Worker that picks the right format (HTML, MD, JSON, YAML, PNG) via headers or paths. Simple, fast, flexible.
Read more in the repo, check the demo, and lmk what you think!"},"story_title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["agents"],"value":"Shared Application Layer for Agents, Crawlers, Humans, and APIs"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/janwilmake/user-agent-router"}},"_tags":["comment","author_janwilmake","story_43484770"],"author":"janwilmake","comment_text":"The web\u2019s stuck serving one format for all\u2014HTML for humans, but what about bots, APIs, or social previews? It\u2019s a legacy mess.
Here\u2019s the thing: one page should adapt\u2014HTML for crawlers, JSON for APIs, Markdown for agents, and PNG for social. Same data, smartly served.
I built a Cloudflare Worker to ditch legacy bloat. Vanilla JS/HTML/CSS, no frameworks, just a shared layer for all clients. First principles.
Meet my User-Agent Router: a Worker that picks the right format (HTML, MD, JSON, YAML, PNG) via headers or paths. Simple, fast, flexible.
Read more in the repo, check the demo, and lmk what you think!","created_at":"2025-03-26T17:47:44Z","created_at_i":1743011264,"objectID":"43484771","parent_id":43484770,"story_id":43484770,"story_title":"Shared Application Layer for Agents, Crawlers, Humans, and APIs","story_url":"https://github.com/janwilmake/user-agent-router","updated_at":"2025-03-26T17:53:23Z"}],"hitsPerPage":20,"nbHits":38,"nbPages":2,"page":0,"params":"query=cloudflare+markdown+agents&advancedSyntax=true&analyticsTags=backend","processingTimeMS":20,"processingTimingsMS":{"_request":{"roundTrip":23},"afterFetch":{"format":{"highlighting":1,"total":2}},"fetch":{"query":12,"scanning":5,"total":18},"total":20},"query":"cloudflare markdown agents","serverTimeMS":22}