Prabal Tripathi

Two Thousand Six Hundred and Seventy-Nine Apps

A test created one record and never deleted it. Then it ran for a year.

A test in our suite creates an application, does its thing, and finishes. It never deletes the application.

That's it. That's the whole bug. One row, left behind, per run.

By the time anyone noticed, the shared environment held 2,679 of them.

Why nobody noticed for a year

Because nothing broke. That's the part worth sitting with.

At fifty records, no difference. At five hundred, the dropdown is longer and nobody cares. At two thousand, a query is slower by an amount that disappears into normal variance. There's no day where it fails. There's no alert for "gradually accumulating."

Then one day someone makes the list virtualised — a completely sensible optimisation, exactly the right call for a list that big — and a test that searches for a specific record starts failing, because the record is now thousands of rows down and no longer rendered in the DOM.

The failure is a year old. It surfaced the week something unrelated changed.

The bit that actually cost us

Triage. Someone looked at a test failing on a virtualised list and reasonably concluded the virtualisation broke it. It didn't — virtualisation was fine. The accumulated junk was the precondition, and it had been building since before anyone on the triage call joined.

That's the real tax on leaked state: not the storage, which is free, but the future debugging session where the cause is a year outside the window anyone thinks to look in.

The fix, and the fix that matters

The immediate fix is a teardown that deletes what the run created. Fine.

The one I care about more is where it goes. Not in each test — someone will write a new test and forget. It belongs at the single point where the record comes into existence, so that creating one automatically registers it for cleanup. Make the right thing the default thing, because the alternative relies on every future person remembering, and they won't. I haven't.

The general version

Anything a test creates, a test should remove. If it can't, something should know about the leak on purpose rather than by accident.

"It's only one row" is true. It's one row times every run since you wrote it.