Everyone quotes the five lies of backtesting. There is a sixth, and it is the most expensive: the fill. A backtest books trades at prices the market would never give you, so it corrupts the return series itself before any metric runs. Here is the cost re-gate I use to tell a real edge from fictional fills.
A hypothetical: the Syrian government receives US$25 billion next month. Could its banks hold the money, its offices turn it into projects, its laws carry the contracts and its people build them, and how long would it take? On the record I keep, the answer is not yet, not at that scale and not quickly, with exceptions the record names: multilateral grants, budget support and operators' own capital.
A data-quality gate that crash-loops on corrupt, stale or all-NaN data is not broken, it is working: the alternative is fail open, trading on garbage with silent defaults. Here is the war-story from both sides, the honest flip where an over-eager gate locked me out of a market, and the one portable rule: on bad data, refuse to trade and alert loudly, never proceed with a default that looks like data.
My earlier book on Syria went on sale the week the World Bank published its estimate of what rebuilding the country would cost. My book said US$250 billion to US$400 billion. The Bank said US$216 billion. Here is what my range was missing, what else did not survive being checked, and why the corrections became a second book.
The Financial Times reports that Syria's Sovereign Fund was valued at US$50 billion last year by the finance minister. The figure traces back to an unpublished ministry estimate relayed by an economist, with no date, method or basis. The only asset value the fund's own officials have given in public is about US$2.5 billion, and its deputy director said in August that its assets were still being counted. Set against the economy and the World Bank's capital stock, damage and reconstruction figures, here is what fifty billion would have to mean, and why, without a stated basis, it cannot yet be checked.
A bar carries one timestamp, but that stamp is ambiguous: is a bar labelled 09:00 the open of the coming interval or the close of the last one? Vendors disagree, and reading the wrong edge is a clean one-bar look-ahead that sails through review. Here is the war-story, why the leak is invisible, and the ingest rule I now enforce on every feed.
Total-return and price-return series look identical on a chart and differ by percent-per-year underneath. Mix them, or fold dividends in twice, and a long-horizon backtest drifts a few percent above the honest one every year and compounds into a phantom edge. Here is the reconstruction check I now run: rebuild total return from price plus the dividend stream yourself and diff it against the vendor's adjusted series.
Survivorship bias is the term every quant knows and almost nobody makes actionable. Here is the one-number diagnostic I run before I trust any cross-sectional backtest: count the dead names as a share of your universe over a decade. If it is near zero, your data quietly deleted the losers and every result is biased upward.
Even a genuine edge dies if you deploy it too large. This is the field note on alpha-side risk: how your own market impact (capacity) and everyone else trading the same signal (crowding) shrink the size at which the edge is still yours, and the crossover test that tells you where your ceiling sits before the market finds it for you.