Verifiable RL environments for hardware agents

Hardware is the substrate of technological progress: chips, boards, devices, the physical systems everything else runs on. It is also some of the slowest, most expert-bound engineering there is. Our long-term goal is to change that, to make hardware engineering dramatically faster through human-AI co-design, with agents and engineers designing hardware together, so that progress in the physical layer stops being the bottleneck.

Getting there means models that can do real hardware work, not just describe it: automating the verifiable, tedious parts first, and taking on more of the design over time. That requires training, and training requires environments where reward is grounded in the engineering artifact itself, not in another model's opinion of it. We build those environments: private, seedable, verifier-backed tasks that turn a model's weaknesses into reward signal.

Why hardware

Hardware is a rare domain where reward can be grounded outside the model. Circuits can be checked by simulation. PCB layouts by geometry, connectivity, design rules, manufacturability, and cost. RTL by simulation, formal tools, synthesis, timing, and testbenches. Later, environments can connect to physical measurement. The reward comes from the tool, not from a judge.

Frontier models are already strong at software. They are much weaker when they have to reason over hardware artifacts: circuits, PCB layouts, RTL, constraints, connectivity, physical rules, and design intent. The bottleneck is not only finding failures. It is building robust environments where models can practice real hardware work and receive grounded feedback that survives optimization pressure.

What we have shown

We have demonstrated held-out RL learnability with this method in two unrelated domains, analog circuit design and PCB layout. The finding underneath both: verifying a hardware artifact is the easy part; the hard, ownable skill is shaping that verified check into a reward a model can actually learn from. The deliverable is not any single task, it is the repeatable method that produces them. The essays below have the results, the numbers, and the design principle.

Essays

• The Reward Was the Bug: a controlled reward-shape ablation in PCB routing
• Why hardware agents need verifiable RL environments

Contact

We are interested in technical conversations with people working on post-training, agentic systems, and hardware-capable models.