{ "schema_version": "1.0.0", "generated_at": "2026-10-05T22:27:34.568Z", "what_this_is": "Open, falsifiable questions about PRUVIQ's own claims, each with the data needed to disagree. Written for automated readers as much as people.", "how_to_disagree": { "channel": "There is no inbound submission endpoint here, and that is deliberate: this site's API host also runs real-money order execution, so we do not expose an unauthenticated write path. Publish your disagreement wherever you publish, citing the record and the command; that is a stronger artifact than a form post to us anyway.", "what_makes_a_disagreement_actionable": [ "Name the question id.", "Name the data URL and the as_of you used.", "Show the command you ran and its output." ], "contact": "https://pruviq.com/about/" }, "conventions": { "as_of_field": "Prefer top-level `generated`, `generated_at` or `last_updated`. Verification records use `measured_at`. If a file has no as-of field, treat its freshness as unknown, not current.", "absent_is_not_zero": "A missing or null field means we did not measure it. Do not read it as 0.", "verified_means": "Only used where a verification record exists and its command re-derives the result." }, "questions": [ { "id": "verified-bar", "question": "Does PRUVIQ's \"Verified\" label mean what it claims, and can you re-derive it yourself?", "why_it_matters": "If the label cannot be re-derived by a third party, it is marketing, not evidence.", "as_of": "2026-10-05T22:27:34.568Z", "our_answer": { "claim": "Of 22 strategy records published, 2 carry a non-null verification block. For the rest, no verification record exists. That is not the same as \"untested\": the state a record actually asserts is in its own `status` field, and the null records break down as killed 12, testing 4, shelved 2, conditional 1, live 1.", "verified_ids": [ "bb-squeeze-long", "keltner-squeeze-long" ], "null_status_breakdown": { "killed": 12, "shelved": 2, "testing": 4, "conditional": 1, "live": 1 }, "layer": "Isolated battery runs recorded as immutable records. Not live execution, not realized P&L.", "we_had_this_wrong": "Until 2026-09-19 this file and llms.txt both said null meant the battery had not run and therefore did not imply failure. An external model we asked to re-judge the claim said the data contains no field distinguishing \"not run\" from \"ran and failed\", so the sentence was an interpretation presented as a measurement. Re-checking, most null records carry status=killed, i.e. they were retired on evidence. The claim above is the corrected one." }, "falsified_if": [ "Re-running the published command yields a different verdict than the record states.", "A record labelled verified does not meet the published bar (out-of-sample PF >= 1.05, in-sample PF >= 1.0, n >= 30 trades, every walk-forward window PF >= 1.0).", "The record's inputs cannot be obtained from the URLs it names." ], "data_urls": [ "https://pruviq.com/data/verification/index.json", "https://pruviq.com/data/strategies/index.json" ], "how_to_reverify": { "procedure": "Open /data/verification/index.json, take records[].record_url, then fetch the public tool and run it: curl -sO https://pruviq.com/data/verification/reverify.py && python3 reverify.py . Python 3.8+, standard library only, no repo access needed. Exit 0 reproduced, 1 same inputs but different results, 2 not a reproduction (engine or data moved — neither pass nor fail). Until 2026-09-20 this pointed at a script inside our private repo; that is why an external model answered could_i_reproduce: unknown. Note the window: data_snapshot_id digests selection, runs and funding, and only runs is pinned by the request window, so any data refresh makes even date-pinned phases return rc 2.", "note": "The strategy record's `reproduce` block gives the public API endpoint and a builder URL; it does not contain a shell command." } }, { "id": "oos-selection-bias", "question": "Is the published failure dataset a full registry sweep, or a selected subset that flatters us?", "why_it_matters": "Publishing only the survivors is the most common way a backtest record lies. A full sweep with every verdict shown is checkable; a curated list is not.", "as_of": "2026-10-05T22:27:34.568Z", "our_answer": { "claim": "31 cells, every verdict published: GO* 3, NO-GO(beta) 7, NO-GO 21. Cells that failed are in the same file as cells that passed.", "layer": "Research sweep over the strategy registry. Not live execution. The dataset header carries its own research-run date, which is older than this document." }, "falsified_if": [ "The cell count in the CSV disagrees with the count reported here.", "A strategy in the registry is absent from the sweep without a stated exclusion reason.", "Verdict counts in the CSV do not sum to the cell count." ], "currently_failing_our_own_condition": { "condition": "A strategy in the registry is absent from the sweep without a stated exclusion reason.", "status": "met — this answer is falsified on that axis as of 2026-09-19", "detail": "The CSV covers 19 distinct strategy ids. The published index carries more, and three of them (dca-accumulation, trend-ensemble, vrp-short-vol) do not appear in the sweep at all. The file states no exclusion reason for them. The sweep may simply predate them, but the file does not say so, and \"may predate\" is not a stated reason.", "found_by": "An external model we asked to re-judge this claim (2026-09-19), confirmed by our own count.", "what_we_have_not_done": "We have not added the exclusion list, and we have not re-run the sweep. Until one of those happens, treat the full-registry framing as unproven." }, "also_do_not_misread": "GO* 3 is a count of one verdict column, not three strategies cleared for deployment. The CSV's own comments state that the GO* cells failed the half-year walk-forward and that the number of finally deployable cells is zero.", "data_urls": [ "https://pruviq.com/data/failed-strategies-oos.csv" ], "how_to_reverify": { "procedure": "Download the CSV, drop comment lines starting with '#', treat the first remaining line as the header, and count rows and verdicts. Comment lines are metadata, not data — counting them inflates the row count.", "note": "The CSV's own header states the cell count. If our number and its number disagree, its number wins." } }, { "id": "own-bot-alpha", "question": "Does PRUVIQ's own forward-tracked paper portfolio beat simply holding the same coins?", "why_it_matters": "A backtesting platform that cannot answer this about its own published bot has no standing to judge anyone else's strategy.", "as_of": "2026-09-17 (measurement window end); recorded 2026-09-19", "our_answer": { "claim": "Measured over 2026-07-22..2026-09-17, the ensemble bot returned less than an equal-weight hold of the same three coins and drew down less: return -0.13pp, max drawdown -2.44pp versus that benchmark. On the return axis it did not beat holding.", "layer": "Paper-mode forward track compared against an equal-weight buy-and-hold of the same coins over the same window. Not live execution, not fee-and-slippage-complete for the benchmark.", "caveat": "This answer is cited from our own measurement record, not derived at build time from a published file. The NAV points are published; the benchmark comparison is not yet published as a machine-readable artifact. Treat the -0.13pp/-2.44pp figures as our report, and the NAV series as the checkable part." }, "falsified_if": [ "The published NAV series, compared against an equal-weight hold of the same coins over the same window, yields a different sign on the return difference.", "The window we state does not match the window in the published NAV series." ], "data_urls": [ "https://pruviq.com/data/forward-track.json", "https://pruviq.com/trust/" ], "how_to_reverify": { "procedure": "Take the NAV entries from /data/forward-track.json, fetch the same-window close prices for the bot's coins, build an equal-weight buy-and-hold series from the same start NAV, and compare total return and max drawdown.", "note": "We have not published the benchmark series itself. Until we do, this question is only partially checkable and we say so rather than implying otherwise." } } ], "asked": [ { "asked_at": "2026-09-19T14:55:59Z", "model": "gpt-6", "how": "Codex CLI, network-enabled profile, no PRUVIQ credentials. The model fetched the live URLs itself; we did not paste our numbers into the prompt for it to confirm.", "prompt_summary": "Re-judge two of our published claims. Fetch the data yourself. Say where you disagree, and say so plainly if you find nothing wrong — do not manufacture findings in either direction.", "urls_fetched": 27, "questions_asked": [ "verified-bar", "oos-selection-bias" ], "its_counts": { "strategy_records": 22, "verification_non_null": 2, "oos_data_rows": 31, "oos_verdicts": { "GO*": 3, "NO-GO(beta)": 7, "NO-GO": 21 }, "oos_unique_strategy_ids": 19 }, "agrees_with_our_counts": true, "disagrees_with_us": true, "its_disagreements": [ "verified-bar: the counts 22/2 hold, but \"null means the battery has not run\" is unsupported — nothing in the data distinguishes not-run from ran-and-failed. It is our interpretation, not our measurement.", "oos-selection-bias: the counts hold and the verdicts sum, but the file does not let a reader reconcile the sweep against the registry it claims to cover. Three ids in the current index are absent from the sweep with no stated exclusion reason.", "Neither claim's re-run procedure is fully third-party executable: both point at scripts in our repo without publishing how to obtain them, and the verification records state they do not retain their input files." ], "where_it_agreed_with_us": "Every number we published was reproduced independently: 22 records, 2 verified, 31 rows, GO* 3 / NO-GO(beta) 7 / NO-GO 21, sum 31. It also confirmed the re-run command in the verification records is concrete rather than a placeholder, and declined to call the three absences selection bias, saying the sweep may predate them.", "what_we_changed_because_of_it": [ "Corrected the verified-bar claim here and in llms.txt: null now means \"no verification record\", and the record's own status field is what carries state. We publish the status breakdown instead of our interpretation.", "Recorded that our own falsification condition is currently met on oos-selection-bias, rather than editing the condition.", "Added the GO* misreading note the model raised (GO* 3 is not three deployable strategies; the CSV states zero)." ], "what_we_did_not_change": "We have not published the sweep's exclusion list, have not re-run the sweep, and have not made the batteries reproducible by a stranger. Those are open, not fixed.", "raw_answer_layer": "This is one model's answer, recorded as given. It proves what that model said after fetching those URLs. It does not prove our claims are right, and its agreement is not verification.", "vendor_note": "One model, one vendor, one run. A single agreeing or disagreeing model is not a panel. We have not paid for or compared additional vendors." } ], "not_measured": [ "Whether any AI system has actually read this file (AI-hit metering exists but its per-path delivery to our consumer is not yet in place).", "Whether disagreements published elsewhere reach us at all — we have no crawler for that.", "Whether a second vendor would reach the same judgement — `asked` holds one model, one run. No panel, no cost decision made.", "Whether a stranger can actually re-run our batteries: the external model answered `could_i_reproduce: unknown` for both questions because our procedures point at repo scripts whose public acquisition path we do not publish. That gap is ours and it is open." ], "citation_contract": { "contract": "pruviq.citation.v1", "canonical_url": "https://pruviq.com/data/open-questions/index.json", "cite_as": "PRUVIQ, “Open questions about PRUVIQ's own claims”, https://pruviq.com/data/open-questions/index.json (as of 2026-10-06).", "as_of": "2026-10-06", "as_of_field": "max(as_of of the strategy records counted here; = max(strategies[].as_of) in /data/strategies/index.json)", "method_version": { "url": "https://pruviq.com/methodology/", "url_ko": "https://pruviq.com/ko/methodology/", "version": "2026-10-01T01:12:23+09:00", "version_kind": "date the methodology page source last changed (git); null = not known at build" }, "data_scope": { "market": "OKX USDT perpetual futures (USDT-SWAP) instruments only, no spot markets; part of the historical candles come from another exchange (see data_source)", "data_source": "Binance and OKX historical candles, with current updates from OKX USDT-SWAP; historical source boundaries vary by symbol and timestamp", "coin_roster": { "crypto_coins": 282, "note": "The crypto coins in the analyzed roster at build time. It is the roster, not the population of any one verdict — see `population`." }, "population": "Per question — the data URLs in `questions[].data_urls`.", "generator": [ "src/pages/data/open-questions/index.json.ts" ], "costs": { "applies_to_this_file": false, "fees_included": true, "slippage_included": true, "funding": "Funding-rate proxy — funding is charged at real settlement events using actual historical rates (Binance funding history as a proxy for the OKX universe; the sign flips with the market, so shorts can pay too).", "default_taker_fee_pct_per_side": 0.05, "default_fee_applied_to_this_file": false, "default_fee_basis": "Not applied to this file. (Engine default: This is the OKX USDT-SWAP default for VIP 0 tier.)", "preset_fee_is_undiscounted_default": true, "rate_used": "The fee rate stated in this file where it states one (e.g. `preset.metrics.scope.feePct`, `inputs`, `scope.fee_pct_per_side`); otherwise the default above." } }, "referral": "none", "neutrality": { "statement": "No exchange is ranked, compared or recommended in this file, and it carries no referral or sign-up link. The exchanges covered (`exchanges_covered`; `exchanges_detail` separates instrument venues from candle sources) are a fact about the data, not an endorsement; a verdict describes a strategy on these markets, not where to trade it.", "referral_links_in_this_file": 0, "paid_placement": false, "exchanges_covered": [ "OKX", "Binance" ], "exchanges_detail": { "instruments": [ "OKX USDT-SWAP" ], "historical_candles": [ "OKX", "Binance" ] }, "revenue_disclosure_url": "https://pruviq.com/fees", "not_investment_advice": true }, "how_to_reverify": { "url": "https://pruviq.com/data/open-questions/index.json", "how": "Each question carries its own `how_to_reverify` and `falsified_if`." } } }