The simple version
A bug you can’t reproduce isn’t a bug that didn’t happen. It’s a bug whose conditions you don’t yet understand well enough to recreate.
Why reproducibility is the wrong goalpost
When a bug report comes in and the engineering team can’t reproduce it, the instinct is to close the ticket. “Cannot reproduce” is a legitimate status in most issue trackers, and there’s a certain logic to it: if you can’t see the problem, you can’t fix it. But this reasoning quietly assumes that the inability to reproduce a bug means the bug was a fluke, a one-off, a ghost in the machine.
It usually isn’t.
Reproducibility is about your understanding of the system, not the system’s behavior. When you can’t reproduce a bug, what you’re actually saying is: “I don’t know which combination of state, timing, inputs, and environment caused this.” That’s a very different statement. One closes the ticket. The other opens an investigation.
The bugs that matter most to users are often the hardest to reproduce in a controlled environment precisely because they emerge from real-world conditions: specific account configurations, network timing, concurrent operations, or a sequence of user actions that no test suite thought to simulate.
Heisenbugs and race conditions
There’s a term in software development, “heisenbug,” for a bug that disappears or changes behavior when you try to observe it. The name is a nod to the Heisenberg uncertainty principle in physics: the act of measurement changes what you’re measuring.
In practice, heisenbugs often involve concurrency. Concurrency is what happens when a program does multiple things at the same time, or appears to. Two threads (independent execution paths) might both try to update the same piece of data simultaneously, and depending on which one gets there first, you get different results. This is called a race condition.
Race conditions are the canonical unreproducible bug. Run the code a thousand times on your laptop and it works fine. Run it under real production load, with hundreds of simultaneous users, and occasionally two requests collide in a way that corrupts data or crashes a process. Then the traffic spike passes, and everything looks normal again.
The reason you can’t reproduce it locally is that you can’t easily recreate the exact timing. And timing, at this level, is measured in microseconds.
Amazon’s Dynamo paper, published in 2007, is largely about building systems that tolerate exactly this kind of inconsistency. The engineers there understood that in a distributed system (one where data is spread across many machines), race conditions aren’t edge cases. They’re the expected operating environment. The paper’s core argument is that you should design for “eventual consistency” rather than pretending these collisions won’t happen.
What unreproducible bugs are really telling you
When a bug can’t be reproduced, it’s usually because one or more of these things is true:
The state matters more than the input. Your test suite probably checks what happens when you pass certain values to a function. It’s less likely to check what happens when those values are processed after a specific sequence of prior operations has left the system in a particular internal state. Real users don’t use software in isolation. They’ve done things before this action, and those things leave traces.
Timing is a variable you’re not controlling. If your bug only appears under load, or only when two things happen within milliseconds of each other, a single-threaded local test will never surface it.
Your environment is too clean. Production has quirks that development machines don’t: specific OS versions, memory constraints, network latency, dependencies that are slightly out of sync. A bug that requires a particular combination of these factors will feel random from outside.
The bug is being suppressed, not fixed. Some systems swallow errors silently. An exception gets caught, logged nowhere useful, and execution continues in a degraded state. The user sees something wrong, but when you look at the logs, there’s nothing there. The bug happened. You just designed your observability (your ability to see what the system is doing internally) to miss it.
That last one deserves emphasis. The bugs that never get filed are often the worst ones, and silent error handling is one of the main reasons they stay invisible.
How to actually investigate these
The engineering reflex for unreproducible bugs is to add logging and wait for it to happen again. That’s not wrong, but it’s incomplete.
Better approaches, roughly in order of what to try first:
Treat the bug report as a data point, not a description. The user said “the page went blank.” That’s the symptom, not the cause. What were they doing for the ten minutes before that? What’s in their account that makes them different from others? What time was it, and was anything else happening on the servers?
Look for correlations in your logs, not just errors. If you have good observability, you can often find the moment the system started behaving strangely even when no error was thrown. Slow queries, unusual memory patterns, retry spikes, these are the fingerprints of a bug that didn’t announce itself.
Chaos engineering, used carefully, can surface timing bugs. Netflix’s Chaos Monkey (a tool that randomly kills servers in production to test resilience) was built on the premise that you need to introduce failure intentionally to understand how your system fails naturally. You can apply the same thinking at a smaller scale: introduce artificial latency between services, restrict memory, hammer a specific endpoint with concurrent requests.
Read the code for assumptions, not just logic. Unreproducible bugs often live in the gap between what the code assumes is true and what’s actually true in production. Look for assumptions about ordering, about atomicity (that two operations will either both succeed or both fail), about exclusive access to shared resources.
Why this matters beyond debugging
There’s a broader principle here. The engineer who fixes the bug rarely knows why it existed, and that knowledge gap tends to compound over time. Systems accumulate unreproducible bugs as technical debt. Each one that gets closed as “cannot reproduce” is a small assumption left unexamined.
The teams that build reliable software are not the ones that write perfect code. They’re the ones that treat every production anomaly as a question worth answering, even when the anomaly doesn’t stick around long enough to be convenient.
A bug that happened once is evidence. Evidence of what, exactly, is the question. Closing the ticket doesn’t make the question go away.
