By this point in the track you can run a backtest and read the result. This article is about the harder question: when do you stop believing one?
Every check below has done real work in retiring a strategy here — not always alone, and not one strategy each. No hypotheticals — each section links to the post where the strategy died, so you can read the full working rather than take this summary on trust.
Check 1 — Does the edge survive its own costs?
The first thing that kills a strategy is not a losing streak. It is arithmetic.
MACD crossover produced “a 50% win rate — a coin flip, exactly — and a negative edge after costs” (Does MACD Work?). Before costs it looked like a wash. With costs applied it was a slow loss.
This is why the exchange article came earlier in this track. A strategy is not “profitable, minus some friction.” The friction is part of the strategy, and a rule that only works when you ignore it does not work.
What to do with it: compute the result with costs on, not as an adjustment afterwards. If the edge only exists at zero cost, you have found a property of your spreadsheet, not of the market.
Check 2 — Is it the settings, or is it the idea?
The most common defence of a failing strategy is that it was tuned wrong. That defence is testable, and testing it is the point.
For Supertrend we ran the whole grid rather than a favourite setting: “0 profitable configurations out of 30” (Does the Supertrend Strategy Work in Crypto?). Not one combination survived.
That result is worth more than any single backtest, in both directions. If 30 settings all fail, “you used the wrong parameters” is answered. And if one of thirty had succeeded, that alone would not have been evidence of an edge — searching thirty options and reporting the best one is a different claim from testing one option and having it work. The more you search, the higher the bar any winner has to clear. We measured how high that bar gets for our own weekly ranking.
What that costs, measured
We put a number on it. Taking the 105 ranking cells in each of four universes over the year to 2026-08-30, we asked what the best of 105 would look like if none of them had any edge at all — and compared our actual best against that bar.
| Universe | Best Sharpe observed | Bar set by picking the best of 105 | Clears it? |
|---|---|---|---|
| BTC | 1.68 | 2.66 | no |
| Top 30 | 1.64 | 2.76 | no |
| Top 100 | 1.52 | 2.75 | no |
| Top 50 | 1.28 | 2.82 | no |
In all four, our best cell does not clear the bar that searching 105 candidates sets on its own.
That bar is not a measure of quality. It is what the maximum of 105 tries tends to reach even when every one of them is worthless — search enough and something always looks good. Our best of the 105 does not get above it, which means we cannot separate the winner from the search that produced it. It does not mean the strategies in this table are worse than noise. It means a rank-1 badge, shown without this correction, is not evidence of an edge.
One detail makes it concrete. In the BTC universe as of 2026-08-30 02:51 UTC, the cell sitting at rank 1 — ranked by profit factor — was Keltner Squeeze SHORT, with a Sharpe of 1.16. It was not even the highest Sharpe among the 105; that was 1.68, and 1.68 does not reach 2.66. It is also the strategy this article’s fourth check declares dead. A live ranking can put a retired strategy on top, and nothing about the badge tells you so.
The figures and the method are in multiple-testing-correction.json, which names its own limits rather than hiding them: the full deflated-Sharpe procedure was not run — the threshold comparison already settled the question — the correlation estimate covers only these four windows, and the trades are not independent.
Two things we want on the record. The correlation adjustment turned out to be almost nothing: 105 candidates behaved like 104.7 independent ones, a shrinkage under 0.3%. And that correlation figure is the one input we cannot show you — it comes from a source artifact that is not published, so an outside reader can check the arithmetic that turns it into the bar, but not the figure itself.
What to do with it: decide the parameter range before you run, run all of it, and report the distribution — not the maximum.
Check 3 — A high win rate is not an edge
Supertrend’s best configuration carried a 57% win rate while losing money (same post — “Why a 57% win rate still loses”).
Win rate says how often you are right. It says nothing about how much you make when right versus how much you lose when wrong. The autopsy does the arithmetic on the actual configuration: “Win 57 trades out of 100 at +4%, lose 43 at −12%, and you are deeply negative before costs …” — a majority of wins, and a losing strategy, because each loss gave back three times what a win banked.
What to do with it: read win rate next to average win and average loss, never alone.
Check 4 — Did the edge do the work, or did the market?
This is the check that has actually reversed a verdict here.
Keltner Squeeze SHORT carried “under test” status when its post went up. An out-of-sample regime test run on 2026-06-28 settled it, and the entry was updated to killed on 2026-08-15: “the SHORT edge is bear-beta. The strategy’s profits in the OOS window came from a falling market, not from the squeeze filter functioning as designed. Adjusted for that regime exposure, no standalone edge survives.” (Does the Keltner Squeeze Work? — the 2026-08-15 update)
A short strategy in a falling market makes money whether or not its entry rule means anything. The question is whether it made more than the market handed it for free.
What to do with it: before crediting a rule, subtract what the direction alone would have earned in that window.
The pattern underneath all four
Each check asks the same question in a different way: is there another explanation for this result? Costs, parameter search, trade asymmetry, market direction — four ways for a number to look like an edge without being one.
That is also why the verdicts here change. Keltner Squeeze was “under test” when its post was published and “killed” a few weeks later; the post kept both, dated, rather than being quietly rewritten. A verdict that can never change was never a measurement.
Where to look next
The strategy library carries a status on every entry, including the dead ones. The autopsies are the long form: MACD, Supertrend, Keltner Squeeze, Momentum Long.
Reading one of those end to end is the exercise. The numbers are ours, the windows are stated, and where a result later changed, the change is on the page.