The short version
Bot sellers like to say a strategy “beats random entries”. Researchers like that test too, and it is a good test. It can also go wrong in a specific way, and it went wrong for us.
We tested a rule we paper-trade, internally called F5-P. It goes long a top-10 USDT perpetual future by dollar volume on a breakout, with a 3% stop-loss, a 20% take-profit and a 48-hour time exit. Against 1,000 sets of random entries it came out at the 1st percentile, and we read that as “the signal is worse than chance”.
The control was wrong. It drew its random entries only on the days F5-P traded, and a breakout later in the same day is what picks those days. A random entry at 03:00 on a day we already know will end in a breakout is not random. It has looked at the future.
When we split the random entries by time of day, the ones placed before the day’s first breakout earned +0.83% per trade. The ones placed at or after it earned −0.14% [−0.59, +0.34], and F5-P itself averaged −0.09% [−0.67, +0.50]. We withdrew the “worse than chance” reading. On 2024-02-01..2026-07-31 both of those intervals are wide and include 0. We did not test the difference between them, so we draw no conclusion that F5-P is better than, worse than or equal to random top-10 longs taken at or after the day’s first signal hour.
Every number below is our own measurement on one window, 2024-02-01 to 2026-07-31, with a fee of 0.05% plus slippage of 0.2% per fill, and funding. The data are hourly USDT perpetual futures from a mixed Binance and OKX history whose source boundaries vary by symbol and timestamp, with Binance funding rates used as a proxy. Our frozen research panel skips trades that span the hours it masks as source changes. It is in-sample. It describes the past on that window and says nothing about future returns.
Weak points first
- The explanation comes from a split, not from a re-run. We split random trades we already had by the hour of the day’s first signal. We did not re-run the original test with a corrected design. Read “the split shows”, not “the corrected test shows”.
- The two random sets are different. The original test used 1,000 draws. The split uses 20 seeds from a separate study with the same exits. The +0.31% below is not a reproduction of the original +0.378%.
- The “after” group is still conditioned on the day. It uses only days on which F5-P fired. After the first signal hour that fact is known, so a live trader could act on it, but it is not a clean unconditional control.
- A second control gives a different picture. Random entries on days chosen without reference to F5-P, with the same exits, lost more than F5-P did (details below). That difference is about +0.55 percentage points per trade, with a confidence interval whose lower end sits at about zero. Both are negative after costs.
- Survivorship. Coins delisted before 2026-06-14 are missing from our data. That flatters long-side results like these.
What a random-entry control is for
A placebo test asks one question: does the strategy’s decision add anything? To answer it, the control should copy everything the strategy does except that decision. Same coins, same exits, same costs, same holding time. Only the entry is replaced by chance.
That is why it is easy to build a control that copies too much. If it also copies which days the strategy traded, and those days were chosen with information the strategy only had later, the control has inherited a piece of the future that the strategy never had.
What we ran
The original control drew 1,000 sets of random trades. Each trade picked a random coin from that day’s top-10 list and a uniformly random hour of that day, then used F5-P’s stop, target, time exit and costs. Over the 529 F5-P trades on this window:
| mean per trade | |
|---|---|
| random control (1,000 draws) | +0.378% |
| F5-P | −0.086% |
| F5-P’s percentile among the random draws | 1st |
An independent review re-checked this arithmetic and found it correct. The arithmetic was right. The design was wrong, and a check of the arithmetic does not test what the control conditions on.
What was wrong
The control’s days were F5-P’s own 274 entry days on this window. F5-P enters when a breakout fires. So every one of those days contains a breakout, by construction, and some of it happens after midnight and before the signal. A random entry early that day gets to ride the move that the breakout later confirms.
The day choice alone moves the result a long way. In the separate 20-seed random study (same 3% / 20% / 48-hour exits):
| random entries | trades | mean per trade [95% CI] |
|---|---|---|
| on F5-P’s days | 2,928 | +0.31% [−0.07, +0.70] |
| on all other days | 6,975 | −0.95% [−1.16, −0.73] |
The split
On F5-P’s days, we split the random entries by whether they were placed before the hour of that day’s first F5-P entry:
| group | trades | mean per trade [95% CI] |
|---|---|---|
| random, before the first signal hour | 1,340 | +0.83% [+0.34, +1.33] |
| random, at or after the first signal hour | 1,588 | −0.14% [−0.59, +0.34] |
| F5-P itself | 529 | −0.09% [−0.67, +0.50] |
The positive result sits in entries made before the breakout that defines the day. A live system cannot know that day in advance. After the signal hour, the random entries and F5-P both have wide intervals that include 0 on 2024-02-01..2026-07-31. We did not run a test of the difference between the two, so this split does not say which is better, or that they are equal.
It is the day, not the hour. A fixed entry at 00:00 UTC every day, the least clever entry there is, looks strong when the day has been picked in hindsight:
| 00:00 UTC entries | trades | mean per trade [95% CI] |
|---|---|---|
| on F5-P’s days | 1,679 | +0.90% [+0.42, +1.38] |
| on all other days | 4,603 | −1.28% [−1.53, −1.01] |
Against a random control that does not look ahead
We also ran random entries at the same rate and with the same exits, on days drawn without reference to F5-P. That control lost −0.577% per trade [−0.771, −0.390], against F5-P’s −0.086% [−0.666, +0.504]. Pooled over 140 seeds, F5-P was ahead by +0.55 percentage points per trade, with a day-paired confidence interval of [+0.004, +1.10]. That lower bound is about zero. A separate diagnostic kept each coin F5-P picked and moved the entry to a random day in the same calendar month, at a random hour. There the gap was +0.30 points [−0.28, +0.91]. That diagnostic uses days later in the month, so it looks ahead itself: it is not tradable and is not a baseline. Read it only as a rough hint that about half of the gap comes from coin choice and half from timing, with neither half clear on its own.
Read the two controls together. Against an untimed random long, F5-P lost less; that comparison is a test of the difference, and its interval sits just above 0. Against random longs taken at or after the first signal hour on the same days, we did not test the difference, so we draw no conclusion either way. In both cases the averages are negative after costs on this window.
The same flaw, a second time
We later found the same mistake in a different yardstick. An earlier comparison held an equal-weight top-10 basket for the whole day on F5-P’s entry days, from 00:00 onward. On those 274 days the basket rose +1.70% between 00:00 and the first entry hour. At the same clock hours across all 912 days on the window it averaged −0.04%. The basket was being paid for the run-up before the breakout, just as the random entries were. That comparison, which had put F5-P behind the basket, is withdrawn.
The yardstick we use now holds the same equal-weight top-10 basket at an exposure reset each day to F5-P’s own average exposure over the previous 30 days. It uses only past information, so a trader could hold it. Two limits apply. In the first 30 days of the window the history is shorter, and on the first day the exposure is zero. And the extra trading cost of changing the basket’s exposure from day to day is not charged; we have not measured it. Against it, F5-P’s difference on this window is +0.008% per day [−0.024, +0.046]. That interval includes zero, so the two cannot be told apart. If you match the basket to F5-P’s exact holding hours instead, the answer runs from −0.042% to +0.021% per day, depending on how the exit hour and the basket’s own costs are counted. We report that version only as a check on coin choice.
For context, that basket itself lost −0.121% per day after costs on this window. Long top-10 exposure was a losing position over these dates. That is a statement about these dates, not a prediction.
A checklist for your own placebo tests
- Where does the control draw from? If it uses the same days or coins as the strategy, ask how those days or coins were chosen, and when that information existed.
- Can the control enter before the information that defined its sample? If a control entry at 03:00 sits on a day chosen by a 15:00 event, it is looking ahead.
- Is there a tradable baseline, not only zero? Compare with a passive basket held at the same average exposure, built only from past information.
- Was the design reviewed, not only the arithmetic? Our arithmetic passed an independent check. The flaw was in what the control was allowed to know.
- Do two controls agree? If a same-day control and an untimed control give different answers, report both.
What this changes, and what it does not
It changes how we read one rule on one window: the “worse than chance” statement is gone, and so is any claim of an edge after costs. It does not say F5-P will win or lose from here, and it says nothing about other strategies. Our forward paper record is where this rule gets judged next, against the 30-day-exposure basket and not against zero or the old random control.
If you want to see how other checks have taken strategies off this site, how we decide a strategy is dead lists them. The verification checklist covers the questions to ask of any backtest, including ours.
Questions people ask
What is look-ahead bias in a backtest? Using information at a time when it did not yet exist. The usual case is in the strategy itself. Here it was in the control: the control’s days were picked by an event that happened later in each day.
Is “beats random entries” a good test? Yes, when the random entries are drawn without knowledge the strategy did not have. Ask how the random days and coins were chosen before you trust the comparison.
Does this mean breakout strategies do not work? It says one breakout rule, on the hourly perpetual-futures panel described above, from 2024-02-01 to 2026-07-31, at a 0.05% fee plus 0.2% slippage per fill and funding, showed no measured edge after costs against a fair control. It does not cover other rules, venues or dates.
Disclosure: our price data are hourly USDT perpetual futures from a mixed Binance and OKX history, with source boundaries that vary by symbol and timestamp; funding uses Binance rates as a proxy. PRUVIQ earns exchange referral commissions; the split is on the fees page. This post carries no referral link. All figures are our own measurements on 2024-02-01..2026-07-31 (in-sample). No third-party source is used.