How far can a coding agent get with your product? AX Check grades the fundamentals and shows real agent sessions.

38 points•0x63_Problems•13 days ago•41 comments•
I'm the co-founder of Gauge, and I built ax-check.com to quickly test how well coding agents can onboard to your product.

You'll get a scorecard, specific suggested fixes, and three full coding sessions that show how agents read your site and use your product.

I built this because similar checks were too noisy. Most suggested obscure technical changes that don't actually make a difference in agent experience (or AX, hence ax-check.com).

This check starts by using DeepSeek 4.1 Flash to try to find key information about your product, starting from the homepage. In actual agent traffic data, we've seen that the key pages are the homepage, llms.txt, pricing, and the docs site (by traffic volume, and by influence), so we focus on those and ignore the rest. We also find that content negotiation for Markdown is legitimately helpful for agents to complete tasks faster and find what they're looking for, so the scan tests that your key pages can serve Markdown.

The other key piece is that we run actual coding agents in sandboxes, and have them try to onboard to your product. You can see the full trace and watch it happen live (we kick it off fresh when you enter a new site). We surface interesting findings like hallucinated URLs, inaccurate docs instructions, or product confusion.

It also detects whether the agents could complete a fully working onboarding autonomously, without being blocked by a login wall. This is still controversial, but I think finding ways to let agents safely onboard autonomously is going to be table stakes within a year for developer tools in particular.

The whole site is agent-friendly itself! You can generally just talk to your coding agent about ax-check.com and it can do the rest. Would really appreciate any feedback to make this useful.

41 comments

brettdav12 days ago
Integrating live bots to demonstrate live visitor experiences is a nice touch. I got an unexpected blocker (`Default Python urllib User-Agent gets 403'd by anc.dev`) I am not able to reproduce yet. Can you expose the scripts that the agents generate?

I took a different route with https://anc.dev (scored 100 on ax-check) and rely on a wider variety of compliance tests. Beyond websites, it also audits binaries (think CLI tooling) to help inclined developers ensure their tools are discoverable by and useful to agentic systems.

0x63_Problems12 days ago
You can reproduce that finding directly from python:

```

>>> request.urlopen("https://anc.dev")

[...traceback info...]

urllib.error.HTTPError: HTTP Error 403: Forbidden

```

In this case (https://agents.withgauge.com/p/runs/40be763c-4678-4677-88df-...) the agent wrote a python script to test the site, and hit the error above. We don't have a full filesystem diff viewer in ax-check.com, although we do have that in the Gauge agents product itself.

brettdav12 days ago
Interesting, thank you. I'll check whether a setting on CF might be interfering. The site checks headers for routing and response types, but it shouldn't affect access.
mastrchief11712 days ago
I've been seeing these pop up recently, I think I saw Mintlify launched one too. Why the sudden increase in attention towards coding agents?
esafak12 days ago
Welcome to HN...
DougN79 days ago
Does it follow redirects? My non-www hostname redirects to the www hostname, but ax-check.com says it gets no response back. Any reason it can't take the hostname as-is instead of stripping off the www?
waterlooalex12 days ago
I tried testing my site syncwith.com and it says its blocked, but its not clear to me why, any ideas? I tested with Claude and it doesn't find any issues like this.

https://www.ax-check.com/syncwith.com

DougN79 days ago
I'm getting the same thing. My non-www redirects to the www domain. Maybe it can't follow redirects?
131712 days ago
it wants a whole domain when the interesting thing i want it to try lives in a directory
0x63_Problems12 days ago
which URL is it? it's kind of a deep assumption since we're evaluating on ease-of-navigation from the homepage, and looking for /llms.txt etc. but there should be a way for me to add support

Read the full thread on Hacker News →

Related stories