When agents design execution algorithms, what actually matters?

We already know an LLM-driven agentic loop can discover better trading strategies on its own — our earlier work with UW–Madison established that it does real research rather than just p-hacking. So feasibility is settled; the open question is the design one. Given that it works, how should you organize the agents — which choices actually move the results, which don't, and by how much? Working with the University of Chicago Project Lab, we ran five different research paradigms against the same backtester and the same three strategy templates and compared them head to head. The pattern that emerged was not the one we'd have guessed: across the dimensions we varied, more was usually worse. More rounds of debate, more context fed back into the loop, more forced agreement between agents — each one eroded the strategies. The setups that won did less: a single round of proposal rather than three, a short curated summary rather than the full transcript, and light, scheduled sharing rather than constant coordination. Sometimes less is more — the island model that finished first on out-of-sample PnL and Sharpe across all three templates won precisely by sharing sparingly and letting its searches run independent the rest of the time.

Every experiment shares one loop. A researcher agent reads context from a working directory, proposes and codes a change to an execution strategy, and hands it to a backtest engine; a local evaluator scores the result, the agent decides whether it is finished or should iterate, and finished runs are committed to GitHub and archived to S3, with heavier backtests farmed out to a self-hosted runner. What changes between experiments is only the research policy wrapped around that loop — how much the agent remembers, whether it argues with itself, whether it edits its own instructions, and whether many copies run in parallel and share what they find.

All five paradigms were tested against the same three execution-strategy templates, each a tunable rule for how to work an order into the market: AFG (aggressor-flow-gate, trade with order-flow pressure), PTG (position-tier-gate, scale participation by inventory), and VRS (vol-regime-sizer, size by the volatility regime). Every result is quoted against the un-evolved base template, and the numbers that decide the contest are out-of-sample: a held-out window the agent never saw while searching.

The first experiment varied only the agent's memory: a metrics-only mode (tens to a few hundred characters per loop), a curated brief-summary mode (one to three thousand), and a full-trace mode (up to a hundred-plus thousand). Within a run, more context did help — the improvement in PnL and Sharpe rises weakly but consistently with the number of characters fed back (OLS \(R^2 \approx 0.17\) for PnL, \(0.19\) for Sharpe). But the best out-of-sample strategies did not come from dumping the full trace back in; they came from the brief-summary mode. Curated memory beat raw volume: a short, well-chosen recap of what worked generalizes better than the entire transcript, which tends to anchor the agent on its own past noise.

The proposer–criticizer setup let two agents argue over a hypothesis for up to three rounds before committing to code. The intuition — more scrutiny, better ideas — did not survive contact with the backtest. Each additional debate round cost roughly 12–33% of PnL versus the base template. A single round of proposal-and-consensus produced the strongest strategies; piling on rounds two and three reliably eroded them (regression slope \(\approx -32\%\) PnL per round). The criticizer talks the proposer out of good, slightly-unusual ideas and toward safe, over-fitted consensus — the multi-agent version of groupthink.

Here the agent could rewrite its own driving prompt. Each cycle has two phases: a research phase that generates, codes, and backtests a hypothesis while writing reasoning traces, and a critique phase that reads those traces and proposes an improved prompt. The safeguard is a hard gate — a rewrite is kept only if it improves at least four of five metrics, and otherwise reverts, so the prompt can ratchet up but never regress. On the VRS template this compounded sharply: Sharpe climbed from \(4.19\) to \(13.72\) and out-of-sample PnL from \(+40.9\%\) to \(+95.3\%\) over the base, the best of which (loop 7) nearly doubled it. Letting the agent improve how it thinks, not just what it tries, paid off — but only because the gate stopped it from talking itself into worse instructions.

The island model ran three populations in parallel — four generations, two loops each — with a migration step between generations that copies the strongest strategies from each island into the others. It won the whole study, and the reason is specific.

Parallelism alone was not the point — sharing was. Run in isolation, the aggressor-flow island was actually losing, at \(-43\%\) versus base; after it imported ideas discovered on the other islands it turned to \(+233\%\). The vol-regime island reached \(+525.8\%\) by its eighth loop, and position-tier \(+30.7\%\). Across all three, every island's largest jump landed in the loop immediately after a migration step. The islands behaved like a real research team: a dead end on one island became another island's biggest win, so sharing failures mattered as much as sharing successes. Out of sample, on a held-out window the agents never searched, all three island bests beat their baselines by \(169\)–\(272\%\).

The last paradigm changes the goal. Instead of optimizing one number, quality-diversity search (MAP-Elites) lays down a grid of behavioral niches — here, how selective a strategy is against how concentrated its trading is in time — and keeps the best strategy found in each cell. The payoff is a map rather than a point: you can see where in strategy space performance lives. On the aggressor-flow arm the best illuminated cell reached \(+214\%\) over base, and PnL peaked at moderate selectivity and moderate timing concentration rather than at any extreme — a structural read on the strategy family that a single-best optimizer would never surface.

Ranking the four optimizing paradigms by their best out-of-sample strategy in each template, the island model is first in every single cell — a clean mean rank of \(1.00\) on both PnL and Sharpe. (Quality-diversity is scored separately: it produces an archive, not a single contender for the global best.)

Out-of-sample best strategy per paradigm and template, with rank in parentheses. The bottom row is mean rank (PnL / Sharpe). The island model ranks first in all six cells.

The through-line is the same tension we keep meeting elsewhere: the best results come from keeping many semi-independent searches alive and sharing between them on a schedule, not from one fast-converging consensus. It is the structure behind the noisy-rewards work, and it is exactly why the economics in how many agents to run favor structured fleets over brute parallelism on the depth-limited problems that real research turns out to be.

Anthony Vo, Elliot Chung, Franklin Liu, Lindsey Lin, Milind Pathiyal
University of Chicago Project Lab & Event Horizon Labs · Spring 2026 (paper in progress)