We’re sharing Unreal Agent — an agent harness that delivers up to 40% cost savings compared to Codex on production workloads and coding/science benchmarks, without any negative performance impact.
125 comments
Fractal tool discovery: tool taxonomy where an agent can "drill deeper" to find what specific tool it's looking for. Helps if/when polluting context with a zillion (mostly unnecessary) tools.
Leveraging splay trees: this is my favorite data structure and I think relatively unused in the context of agents/harnesses. A lot of times, recently-used workflows/tool-chains will be used again, so having those at the top of the search hierarchy is an awesome optimization.
Virtual containerized notebooks: models working in sandboxed (WASI) Python notebooks is incredible. Even local models (if given enough time) will usually converge on a good solution. Being able to mount tools/resources/fs is again, imo quite untapped. Some problems here are running native things (thing numpy/pandas) in containers is a nightmare (or impossible).
Anyway, happy to see other folks seriously doing stuff in this space. If anyone wants to collaborate on anything don't hesitate to reach out :) I'm also actively looking for a job or some contract gigs.
Fun times ahead.
Ugh, I can't help but respond to this one point. The fact that this is even an issue in the current year just tells us how screwed the software field is in a lot of ways. I don't mean that in existential terms, but of how divided we've become in terms of what's happened to human reasoning. On the one hand, you have people who apply deep thinking to develop the sort of approaches you described, and there's the exponentially growing segment of not-even-programmers who seem to never ask themselves whether any of their ideas have any sort of consequences.
Take MCPs for instance. Sure, I guess it can sometimes make sense to have a stateful API that is optimized for agents. Yet, more often than not, these MCPs frontload a ton of context where it's not needed, and solve problems where none existed. Merely sticking an API (MCP) in front of an API (CLI, REST, GraphQL) without a benefit that can be explained in a single sentence is lunacy and demonstrates a real lack of complex thinking.
But, you just very clearly described why human software engineers are in higher demand than ever before.
It's always been "screwed" in the sense that everyone else sucks at wielding the power.
What we're actually witnessing is a watershed moment where a lot of technologically illiterate people are getting left behind. Those of us on the literate side are left to fight amongst ourselves and the powers that be for control over the future. We have actually been doing an alright job all things considered (else this conversation wouldn't be happening). Politicians are aging out and we're the adults in the room now.
However, MCPs are a godsend to anyone trying to manage access to systems (especially SaaS systems) in a large organization that includes a ton of non-technical employees. We use at least a couple of SaaS products where RBAC is severely limited in what it can govern or where API keys do not have any kind of governance so handing one out is basically handing out read/write on the entire system. Being able to configure an MCP to only have read endpoints enabled it a very useful piece of duct tape. Obviously it would be great if the underlying API was just designed with proper governance but that's out of the user's control.
mcp with 2 tools: execute(code) and search(query)
all other tools, mcps, apis, whatever are encapsulated by the one interface. new tools don't bloat the agent's context, and it can write its own code to perform more advanced and batch operations against the available tools (executed within a sandbox).
executor is a great implementation of this - https://executor.sh
opencode v2 also provides its own native implementation
For some reason they're comparing their harness running on Astra xhigh to Codex with Astra max?
---
Also worth noting that OpenAI just added support for async tool calling to their harness, which isn't 1:1 with this approach, but is slowly ramping up in being able to provide something similar.
A big part of why Codex uses so many tokens is that it basically hot loops on polling tasks it starts for... absolutely no good reason: https://www.reddit.com/r/codex/comments/1wdlp7q/weve_discove...
I fixed it on my fork of Codex too, also back in Jan/Feb – I keep this patch rebased, for anyone who wants it: https://github.com/tekacs/codex/commit/9ffcf8db9078eae43d411...
It results in token savings similar in scale to those displayed here by Unreal.
---
My harness has used a slightly fancier version of the approach that Unreal is using since ~Feb, and... it definitely works excellently, but it's also assuredly smoother with Astra and other recent models that are more aware of async tool calling.
[features.multi_agent_v2]
enabled = true
wait_agent_enabled = true
min_wait_timeout_ms = 10000 # 10s
default_wait_timeout_ms = 300000 # 5m
max_wait_timeout_ms = 3600000 # 1h
each individual benchmark that is combined into agentic coding index was compared on xhigh between ua / codex / pi, and headline improvement was calculated on xhigh, but then agentic coding index pareto chart by default includex codex max, hense the confusion.
This plainly seems like a trademark issue in progress considering it's in the same exact domain and considering how many others were confused the way I probably was.
People, I'm begging you, please talk to a lawyer before launching.
>Computer software, namely, game engine software for video game development and operation; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of video games; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of content for virtual worlds and 3D platforms; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of motion pictures, television shows, videos, 3D animations, 3D simulations, 3D visualizations, virtual reality motion pictures, virtual reality television shows; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of virtual reality video games; Virtual reality game software; Virtual reality software for creating multimedia content; Augmented reality game software; Augmented reality software for use in mobile devices for integrating electronic data with real world environments for the purposes of entertainment
Sure you can, trademarks are contextual. If they were a landscaping business it wouldn't matter. But within the same industry absolutely.
https://github.com/Pyrolistical/pi-notify
Now my pi agent setups its own trigger to notify itself when a background process is done
Would be interesting to compare to a harness optimizing for cost reduction too.
Read the full thread on Hacker News →
Related stories
- Hacker News · 1 points · 2 days ago
- Hacker News · 421 points · 6 days ago
- Hacker News · 3 points · 3 days ago
- Launch HN: Vespper (YC F24) – SOTA Docx MCPvespper.comHacker News · 31 points · 2 days ago
- Hacker News · 1 points · 6 days ago
- Hacker News · 1 points · 4 days ago