I have written before about the failure mode that worries me most in software: a system that returns a well formed answer that happens to be false. Not a crash, not a stack trace, not a NaN. A number with the right units and a plausible magnitude, sitting in a field on a screen, wrong.
I spent a day extending the fireball solver in that direction on purpose. It now goes outward from an observation to the heliocentric orbit the rock was on, and inward from there to breakup, to a fragment cloud, to a list of 500 metre squares with a probability attached to each. Real orbital mechanics, real atmospheric physics, all of it established and none of it invented by me.
The interesting part is not any of that. It is that three separate times the physics came back and contradicted the thing I had built it to show. Each one was more useful than the feature I had set out to add, and none of them would have surfaced without a specific kind of test.
One: the width of a strewn field does not come from breakup
When a meteorite falls apart and the pieces reach the ground, they land in a long thin ellipse. The length is easy to explain. Heavier fragments have a higher ballistic coefficient, they punch through air better, and they carry further downrange. So the field sorts itself by mass along the direction of travel.
The width has always bothered me. The solver draws a search ellipse, and until this week the width of that ellipse was not derived from anything. It was a stated question: where could this land if the crosswind is off by ten metres per second. Honest, but not a prediction.
So I built a fragmentation model to replace it. A body comes apart when ram pressure exceeds its strength. The pieces then push each other apart aerodynamically, sitting in each other's wakes, and that transverse kick is a real effect with a real derivation. That kick, I assumed, is where the width of a strewn field comes from.
It is not. The derivation gives a separation speed of
v_T = v sqrt(2 C rho_air / rho_body)
and the thing that sets the width is not that speed but its ratio to the forward speed, which is what that square root is. In the thin air where meteoroids actually break, around 40 km, it comes to about one part in a thousand. Carried down the remaining path, that is a couple of hundred metres.
I measured it on the worked example: 230 metres of width against a 10 kilometre length. Then I checked whether I had cheated by applying the kick too low, and moved it up to the breakup altitude, where the pieces are still doing 15.7 km/s. That made it narrower, 100 metres, because the air up there is thinner still and the ratio is smaller.
The crosswind term I was trying to replace produces 13 kilometres on the same fall.
Fifty to one, in favour of the thing that was already there. The correct conclusion is not the one I set out to demonstrate. It is that the crosswind question was the right question all along, and that anyone walking a field should spend their effort on getting a wind sounding rather than on modelling a breakup. That ratio is now pinned by a test, because the obvious way for this to go bad later is somebody tuning the interaction coefficient until the numbers agree with what was hoped for.
What the fragmentation model does earn its place for is the other axis: how many pieces, how heavy, how far apart along the track, and which squares to walk first. That part is genuinely new. The width was never going to be.
Two: there were two Earths, and one of them was wrong
To work out what orbit a meteoroid was on before Earth caught it, you need to know where Earth was. That sounds like a solved problem, and it is, but I had solved it twice in the same codebase without noticing.
The backtracking code carried its own copy of Earth's orbital elements, written months ago. So I built a general planet table, all eight, so the same machinery could answer questions about Mars or Jupiter. Then I did the thing that catches this class of bug: I compared the new Earth against the old one.
They disagreed by a million kilometres.
The old code advanced Earth's mean anomaly at 0.9856474 degrees per day. That is the tropical year, 365.2422 days, the one that runs from equinox to equinox and governs the calendar. Mean anomaly measures time since perihelion, and that is the anomalistic year, 365.2596 days. The two differ by about twenty five minutes, because the tropical year is tied to the slowly precessing equinox rather than to the shape of the orbit.
Twenty five minutes a year is 0.0172 degrees. By 2025 that is a third of a degree. By 2050 it is 0.86 degrees, which at one astronomical unit is over two million kilometres of Earth in the wrong place, and roughly 200 m/s in the velocity that gets subtracted from every meteoroid the code ever sees.
Here is the part that matters. Nothing broke. The orbit shape was right. Earth's perihelion distance came out at 0.9833 AU, correct. Aphelion 1.0167, correct. Perihelion still landed in early January, correct. Every test I had written for that file passed, because every one of them tested the shape of the orbit and none of them tested its phase. There was no symptom. The only thing that ever revealed it was building a second, independent Earth and asking whether the two agreed.
I have started treating that as a rule rather than a lucky catch. If a quantity matters, and you can compute it a second way that shares no code with the first, do that and compare. It is the only check that catches an error living in the formulation itself. Conserved quantities cannot do it, because the wrong orbit conserves energy perfectly.
Three: which measurement to fix is not a fixed answer
The last piece of the day was an uncertainty budget. Every number the tool produces is a single answer, and a single answer with no error bar tells you where to stand, not where to walk. So it now samples every input together, runs the entire chain a few hundred times, and reports two things: a probability map over ground cells, and a ranked list of which input is costing you the most search area.
By this point I had a confident opinion, because finding number one had just taught me that wind dominates. I wrote that in the plan. Then I ran it.
With a careful eyewitness reporting angles to about a degree, the angles are 55 percent of the total error and the wind is a distant fourth. The advice "go get a wind sounding" is wrong for that person. The advice is go get a calibrated camera.
Feed the same fireball in with camera-grade angles, 0.05 degrees, and those two bars drop by a factor of twenty, to 6 percent between them, and nothing else moves at all. Now the top of the list is the surviving mass, which the tool does not even predict, and wind direction right behind it. The ranking has inverted, and so has the recommendation.
So my generalisation from finding number one was wrong too, or rather it was right only for a case I had not noticed I was assuming. A tool that always names the same culprit is decoration. The reason to compute the budget per case is precisely that the answer moves.
Two things in that chart look like bugs and are not, which took me a moment each.
Timing contributes nothing. Not a small amount, nothing, exactly. How long the fireball was visible sets the entry speed, and the entry speed decides what the object was and whether it is worth searching for at all. But it does not move the terminal point and it does not touch the dark flight, so it cannot move where the rock lands. A stopwatch buys you an orbit, not a smaller search.
With no wind entered, the wind direction row is also zero, while wind speed is not. A wind of zero has no direction to be uncertain about. That one did start as a real defect: the speed uncertainty was spreading the possible impacts along a single line pointing at a nominal direction nobody had ever measured, when the honest shape is a disc. Caught it because the app showed "nothing" where the library had shown two kilometres, which is exactly the kind of disagreement worth chasing.
What actually made these findable
Six hundred and twenty seven assertions now, which sounds like diligence and mostly is not. Most of them are cheap. The ones that did the work fall into three kinds, and I would keep those three at the expense of all the rest.
Things with known answers. Kepler's third law gives a planet's period from its semi-major axis. My planet table carries the semi-major axis and an independently observed mean motion, in separate columns, and physics says those are not independent. Computing the period both ways for all eight planets is a check on the transcription that does not depend on me having transcribed it correctly. Re-reading the table against the page I copied it from would prove nothing, because that is the same act that could have gone wrong.
Two implementations that share no code. The closed-form conic propagator versus the RK4 integrator, agreeing to under a metre over 24 hours. The new Earth versus the old Earth. This is the only class of test that catches a wrong formulation.
Scaling laws rather than values. The separation speed derivation claims v_T goes as the speed, as the square root of air density, as one over the square root of body density, and not at all with fragment size. Each one is a test that varies a single input and watches the output. A scaling test catches an algebra slip that a single-point comparison sails straight past, because a wrong formula can still hit one right number.
None of the three findings came from a test I wrote to find them. They came from tests written to check something adjacent, which then disagreed with what I expected to see. The day's actual lesson is that the value of a test is not that it passes.
One more, which I nearly shipped
There is a fourth I will mention because it is the most embarrassing and the most instructive.
The breakup model needs the speed of the object at each altitude, and I had two measurements: entry speed at 82 km, terminal speed at 21 km. I interpolated linearly between them. It is the obvious thing to do and it is badly wrong, because deceleration is violently back-loaded. Drag goes as density times speed squared, density climbs roughly exponentially on the way down, and a real body holds most of its entry speed until quite late and then dumps it.
A straight line puts the object at 10.5 km/s in the middle of its flight. The correct answer is that it is still doing 16.7 of its 17. That is a factor of 1.6 in speed and, since pressure goes as the square, 2.5 in ram pressure, and it put my predicted breakup altitude at 23.9 km when the observed band for a stony body is 25 to 50.
The only reason I caught it is that I had written the test as "breakup lands inside the band where breakups are actually observed" rather than as "breakup equals whatever my code currently produces." The first kind of test can tell you that you are wrong. The second kind can only tell you that you have changed.
Replacing the straight line with the classical no-ablation drag solution, fitted through both measured endpoints so the two ends stay measurements and only the shape between them is modelled, moves it to 39.7 km. Which is where it should have been.
The tool is called rainshower and it is not public yet. The physics in it is all standard and none of it is mine: plane intersection from the 1980s, the US Standard Atmosphere, Kepler, Tisserand, ram pressure against strength. What is worth taking from a day of building it is not the software. It is that three separate times a test disagreed with what I expected, each disagreement was more useful than the feature it interrupted, and none of them announced themselves. They had to be gone looking for.
The arithmetic is kept as plain modules with no dependencies and no build step, so the same files run in a browser, in a server route, and under bare node. That is not a style preference. When the product is a number, a calculation nobody can rerun is a calculation nobody should believe, and I would rather that constraint be structural than aspirational.