Losing to a linear network
In the second half of September I lost every game that mattered. Not me: my networks. Every neural network with hidden layers I trained for Falcon, my chess engine, lost to the one already in production, a much simpler linear network. And not by a little: some lost by 200 Elo, others by more than 500.
The odd thing is that it should have been the other way round. This post is about how I ruled out the suspects one by one, and what was really going on. Spoiler: the main problem wasn’t in the networks.
Linear vs. deep
Falcon evaluates positions with an NNUE, a neural network designed to run fast on a CPU inside the search. It has two parts. The first, the feature transformer (FT), turns the position into a large vector. It’s the expensive part, but it’s updated incrementally move by move. The second part is whatever comes after it.
In a linear network, nothing comes after: a weighted sum of the vector and you have the evaluation. In a deep network, the vector first goes through a couple of small layers that can learn relationships a sum can’t capture. Top engines, Stockfish included, use hidden layers. My production network was linear, with 1024 neurons in the FT, and the plan was simple: add a deep head and gain Elo.
Three terms to follow the rest:
- Elo: the strength difference between two versions. +20 means the new one wins a bit more than it loses; −500 means it doesn’t stand a chance.
- SPRT: a statistical test that keeps playing games between two versions until it can decide with confidence whether the new one is better. It’s the final judge.
- Nodes vs. time: you can match two engines by giving them the same time per move or the same number of nodes (positions examined). With time, speed counts; with fixed nodes, only the quality of the evaluation does.
The results
Until 26 September I trained on a laptop GPU, a GTX 950M, so every network was a bet of several days. The numbers, against the linear network:
| Deep network | Result |
|---|---|
| 16 → 32 → 1 head with TD target | −131 at time control; −177 with the best recipe |
| Same, with WDL 0.3 result blending | −287 |
| HalfKA inputs, four different recipes | between −207 and −221 |
| Wide 2048 → 32 → 1 head | between −482 and −520 |
When everything loses, it’s tempting to conclude the idea is bad. But deep networks work in other engines, so something of mine had to be wrong.
The list of suspects
1. Is the engine evaluating wrongly?
If the engine computed the network differently from the trainer, any good network would look bad. I checked by running the network’s forward pass in floating point straight from the weights the trainer saves, and comparing it with the engine’s output.
Verdict: the engine was fine. What was wrong were two Python verification tools I’d used before: they read the weights without the transposition the save format applies. For a while I’d been comparing the engine against a wrong reference.
2. Compressed range?
In very unbalanced positions, the deep network gave evaluations about five times smaller than the linear one. It looked “timid”. I tried expanding its range with a monotonic transform at inference time, without retraining.
Verdict: it lost 49.5 Elo in an SPRT. The compression was real, but it wasn’t the cause.
3. The training sigmoid?
The network learned little in the 1000–2000 centipawn band, and the suspect was the sigmoid that training uses to turn evaluations into probabilities. I doubled its scale, from 400 to 800.
Verdict: the metric was fixed completely (correlation in that band went from 0.13 to 0.87) and the Elo didn’t move: −263. It was the first hint of what became a lesson: static quality metrics don’t predict Elo.
4. Weight clipping?
Going through the trainer (bullet), I found that its default settings clip every weight to ±1.98, including the layers stored in floating point that don’t need it. The 99th percentile of one of those layers was 1.979: the weights were literally pinned to the ceiling.
Verdict: a real defect. I removed it and the Elo stayed the same.
5. Saturated neurons?
The first hidden layer’s activations saturated a lot. I added an extra linear neuron to give the network a path without saturation.
Verdict: saturation went down and the compression stayed identical. A symptom, not the cause.
6. Inherited search margins?
An engine’s search has pruning margins tuned to the scale of its evaluation, and I was using the margins tuned for the old network with the new ones.
Verdict: it mattered, a lot: about 110 Elo. But it still didn’t explain the deficit.
7. The culprit: go nodes
To compare two networks on quality alone, without speed counting, I matched them at fixed nodes: each engine examines exactly N positions per move. A deep network is slower, but at fixed nodes that shouldn’t matter.
On 27 September I found that Falcon’s UCI protocol parser ignored the go nodes command. The engine received “search 25,000 nodes”, ignored it and played on time. All my “fixed-node” matches had been played on time, and so they mixed quality and speed. I’d spent weeks trying to fix the quality of networks whose problem was, to a large extent, something else.
What was really going on
With go nodes fixed, I repeated the measurement with a replica of bullet’s public deep-network recipe:
| Pace | Replica vs. linear network |
|---|---|
| 5,000 nodes per move | −21.3 |
| 25,000 nodes | +2.0 |
| 100,000 nodes | +25.7 |
| On time (40 s + 0.4 s) | −17.7 |
Hover or tap a point to read its value.
This table explains everything. The deep network is better, and it scales better the more the engine searches: with more nodes, its advantage grows. What held it back at real time controls was speed. Each position costs more to evaluate, so in the same time it examines fewer positions and gives back what it gained in quality.
And in my case speed weighs more than in other engines. Falcon plays on a laptop with an i5-7300U and only 3 MiB of L3 cache, which limits how much the network can grow before everything slows down.
Towards r2
With the diagnosis clear, the path was to widen the FT and slim down the head. A version with a 1536-wide FT beat the replica (+32.2 at 25,000 nodes, +37.5 at 100,000), but unoptimized it ran at a fifth of the linear network’s speed. I cut the first hidden layer down to just 8 neurons: 1536 → 8 → 32 → 1.
The first training run came out broken: one of the network’s eight output groups had all its neurons dead from very early on, and the network lost 594 Elo. I retrained it with a different random seed, and out came r2:
- +44.4 at 25,000 nodes and +62.0 at 100,000 against the linear network.
- +16.9 ±7.5 in an SPRT at real time control (8 s + 0.08 s, 2,572 games).
r2 went into production on 30 September, with an optimized AVX2 inference that gives exactly the same results as the reference one. It was Falcon’s first deep network to beat the linear one where it counts: in real play. The dead neurons deserve a post of their own, and it’s the next one in the series.
Why it took so long
There wasn’t one defect: there were several, stacked on top of each other. Some were real (the weight clipping, the margins, the verification tools) and fixing them was right, but none of them was the cause. Every fix that didn’t move the Elo looked like more proof that deep networks just didn’t work for me, when in fact I was measuring wrong.
Three lessons I’m taking away:
- Check that the measurement measures what it says. A protocol bug turned “fixed nodes” into “fixed time” without any warning.
- Verify against the trainer, not against another tool. If two implementations share the same mistake, comparing them proves nothing.
- The game is the judge. Quality metrics improved in networks that went on to lose 250 Elo.
Since then the deep head has kept improving, with a dual activation (+6.3) and a longer training tail (+3.7). But that’s another story.