We're adding browser state persistence soon, hoping to enable it so once you sign in with google once, it can stay signed in on your local machine.","title":null,"type":"comment","url":null}],"created_at":"2025-04-28T16:18:54.000Z","created_at_i":1745857134,"id":43823137,"options":[],"parent_id":43822659,"points":null,"story_id":43822659,"text":"This is very cool! Does your MCP server preserve cookies/localStorage between steps, or would developers need to manually script auth handshakes?","title":null,"type":"comment","url":null},{"author":"esafak","children":[{"author":"neversettles","children":[],"created_at":"2025-04-28T16:39:11.000Z","created_at_i":1745858351,"id":43823341,"options":[],"parent_id":43823160,"points":null,"story_id":43822659,"text":"We started with using browser-use because they had the best evals: https://browser-use.com/posts/sota-technical-report
- but we found that Laminar came out with a better browser agent (& a better eval): https://www.lmnr.ai/ so we're looking to migrate over soon!","title":null,"type":"comment","url":null}],"created_at":"2025-04-28T16:20:46.000Z","created_at_i":1745857246,"id":43823160,"options":[],"parent_id":43822659,"points":null,"story_id":43822659,"text":"Is there a benchmark for this? If not, you ought to (crowd?)start one for everybody's sake.","title":null,"type":"comment","url":null},{"author":"nico","children":[{"author":"neversettles","children":[],"created_at":"2025-04-28T16:45:37.000Z","created_at_i":1745858737,"id":43823403,"options":[],"parent_id":43823280,"points":null,"story_id":43822659,"text":"In browser MCP, looks like cursor controls each action along the way, but actually what we wanted was a single browser agent that had a high quality eval that could perform all the actions independently (browser-use)","title":null,"type":"comment","url":null}],"created_at":"2025-04-28T16:32:13.000Z","created_at_i":1745857933,"id":43823280,"options":[],"parent_id":43822659,"points":null,"story_id":43822659,"text":"Looks amazing. Congrats on the release
How does this compare to browser mcp (https://browsermcp.io/)?","title":null,"type":"comment","url":null},{"author":"proc0","children":[{"author":"neversettles","children":[],"created_at":"2025-04-28T18:55:38.000Z","created_at_i":1745866538,"id":43824731,"options":[],"parent_id":43824256,"points":null,"story_id":43822659,"text":"Yeah, I suppose to esafak's point, perhaps a benchmark for browser agent QA testing would be needed.","title":null,"type":"comment","url":null}],"created_at":"2025-04-28T18:08:07.000Z","created_at_i":1745863687,"id":43824256,"options":[],"parent_id":43822659,"points":null,"story_id":43822659,"text":"Interesting. I see from the video example it took a lot of steps and there is a lot of output for a simple task. I'm thinking this probably doesn't scale very well and more complex tasks might have performance challenges. I do think it's the right direction for AI coding.","title":null,"type":"comment","url":null},{"author":"klntsky","children":[{"author":"neversettles","children":[],"created_at":"2025-04-29T06:01:16.000Z","created_at_i":1745906476,"id":43829114,"options":[],"parent_id":43828361,"points":null,"story_id":43822659,"text":"The power here is the coding agent has the ability to test visually if - and like a human would. So if the button isn't visible, the browser agent would use vision to detect that it's missing.
It sorta tests 'just like a human would' to make sure the flow that's implemented works as it's expecting to.","title":null,"type":"comment","url":null}],"created_at":"2025-04-29T03:07:37.000Z","created_at_i":1745896057,"id":43828361,"options":[],"parent_id":43822659,"points":null,"story_id":43822659,"text":"I told windsurf to install playwright, identify crucial workflows of the app and add tests for them. Not without my input, but I got what I wanted without getting the hands dirty.
Does this thing add much on top?","title":null,"type":"comment","url":null},{"author":"gitroom","children":[],"created_at":"2025-04-29T08:12:47.000Z","created_at_i":1745914367,"id":43829889,"options":[],"parent_id":43822659,"points":null,"story_id":43822659,"text":"Gotta say, getting rid of all the clicking and checking just sounds like a huge win. I hate wasting time on all that.","title":null,"type":"comment","url":null}],"created_at":"2025-04-28T15:36:25.000Z","created_at_i":1745854585,"id":43822659,"options":[],"parent_id":null,"points":84,"story_id":43822659,"text":"Hey HN! We\u2019ve been building an MCP server to help AI-assisted web app developers by using browser agents to test whether changes made by an AI inside an editor actually work. We've been testing it on scenarios like verifying new flows in a UI, or checking that sending a chat request triggers a response. The idea is to let your coding agent both code and evaluate if what it did was correct. Here\u2019s a short demo with Cursor: https://www.youtube.com/watch?v=_AoQK-bwR0w
When building apps, we found the hardest part of AI-assisted coding isn\u2019t the coding\u2014it\u2019s tedious point-and-click testing to see if things work. We got tired of this loop: open the app, click through flows, stare at the network tab, copy console errors to the editor, repeat. It felt obvious this should be AI-assisted too. If you can vibe-code, you should be able to vibe-test!
Some agents like Cline and Windsurf have browser integrations, but Cline\u2019s (via Anthropic Computer Use) felt slow and only reported console logs, and Windsurf\u2019s didn\u2019t work reliably yet. We got so tired of manually testing that we decided to fix it.
Our MCP server sits between your IDE agent (Cursor/Windsurf/Cline/Continue) and a Playwright-powered browser-use agent. It spins up the browser, navigates your app per instructions from the IDE agent, and sends back steps, console events, and network events so the IDE agent can assess the app\u2019s state.
We proxy Browser-use\u2019s original Claude calls and swap in Gemini Flash 2.0, cutting latency from ~8s \u2192 ~3s per step. We also cap console/network logs at 10,000 characters to stay within context limits, and filter out irrelevant logs (e.g., noisy XHR requests).
At the end, the browser agent outputs a summary like:
Web Evaluation Report for http://localhost:5173 \n Task: delete an API key and evaluate UX\n Steps: Home \u2192 Login \u2192 API Keys \u2192 Create Key \u2192 Delete Key\n Flow tested successfully; UX had problems X, Y, Z...\n Console (8)... Network (13)... Timeline of events (57) \u2026\nTry it in Cursor / Cline / Windsurf / Claude Desktop: (macOS/Linux):
curl -LSf https://operative.sh/install.sh -o install.sh\n less -N install.sh # inspect if you\u2019d like\n bash install.sh # installs uv + jq + Playwright + server\n # then in Cursor/Cline/Windsurf/Continue: craft a prompt using the web_eval_agent tool\nWhat we want to do next: pause/go for OAuth screens; save/load browser auth states; Playwright step recording for automated test creation and regression test creation; supporting Loveable / v0 / Bolt.new sites by offering a web version.
We\u2019d love to hear your feedback, especially if you\u2019ve experienced the pain of having to manually test changes happening in your web apps after making changes from inside your IDE, or if you\u2019ve tried any alternative MCP tools for this that have worked well.
Try it out if you feel it\u2019d be helpful for your workflow: https://github.com/Operative-Sh/web-eval-agent. (note: the server hits our operative.sh proxy to cover Gemini tokens. The MCP server itself is OSS; Anthropic base-URL support is coming soon. Free tier included; heavy users can grab the $10 plan to offset our model bill.)
Let us know what you think! Thanks for reading!","title":"Show HN: Web-eval-agent \u2013 Let the coding agent debug itself","type":"story","url":"https://github.com/Operative-Sh/web-eval-agent"}