# Research Ledger — what was tested, what worked, what failed

**Trigger word for any Cursor chat:** `LEDGERIT` (alias `/ledger`)

Say **LEDGERIT** when you want the agent to flush this session’s tests/insights into the ledger. The always-on rule `.cursor/rules/research-ledger.mdc` also requires agents to append after canonical backtests without being asked.

**Purpose:** Durable memory across Cursor sessions. Agents and humans consult this before proposing “new” research, and **append a row** after any meaningful backtest (success *or* failure).

| Artifact | Role |
|----------|------|
| This file | Narrative, update rules, methodology lessons |
| [`research_ledger.csv`](research_ledger.csv) | Machine-readable DB of tests (source of truth for status/metrics) |
| [`research_ledger_cli.py`](research_ledger_cli.py) | `summary` / `list` / `search` / `trades` / `append` |
| [`research_ledger_trade_stats.csv`](research_ledger_trade_stats.csv) | Trade-log aggregates (`tl_*`) |
| [`BEST_IDEAS.md`](BEST_IDEAS.md) | Current preferred *live/research* book (not a full failure history) |

**Last bulk seed:** 2026-07-18 (transcripts + QS rank-50 + Chan suite + overnight VIX + catalogs).

---

## How to use (agents)

1. **Before** proposing a new strategy or re-running an old idea: skim `research_ledger.csv` for matching `token` / `family` / `status=fail|weak`.
2. **After** a canonical or long backtest: append one CSV row + optionally a short note below under “Session notes”.
3. Prefer **updating an existing `id`** if you re-ran the same config with better metrics; set `verified_date` to today.
4. Do **not** cite `nav_q` returns as live-account returns. Do **not** use exit-day-only option metrics for headline DD/Sharpe (use `evaluate_theta_margin`).

### Status vocabulary

| `status` | Meaning |
|----------|---------|
| `default_book` | In maintainer preferred stack (options and/or stock-only) |
| `optional` | Positive evidence; satellite / opt-in flag |
| `promising` | Positive niche evidence; not yet in book |
| `weak` | Positive but lags alternatives or regime-sensitive |
| `fail` | Negative / broken OOS / abandoned |
| `deprecated` | Replaced by a better implementation |
| `research_only` | Useful for upper-bound comparisons only |
| `ideation_only` | Catalogued / brainstormed — **not backtested** as that list |

### Metric caveats (always fill `metric_caveat`)

| Tag | Meaning |
|-----|---------|
| `mtm_margin` | Trust for options DD/Sharpe |
| `exit_day` | Understates option risk |
| `nav_q` | Research upper bound; multi-sleeve full notional |
| `fund_mode` | Single-account fractional weights |
| `paper_era` | Long / different sample than tradable ETFs |
| `in_sample` | No clean OOS claim |
| `tail_driven` | Mean return dominated by rare days |
| `no_costs` | Yahoo/ETF backtest without commissions |
| `lookahead` | Uses future information — not tradable |
| `ideation_only` | No backtest metrics — catalog only |
| `trade_log` | Metrics from closed trades (Σpnl, WR, etc.) |
| `exit_step_sharpe` | Sharpe on equity that steps only on exits — often inflated vs daily MTM |
| `pnl_pct` | Per-trade return fraction (not $ PnL); do not treat Σ as capital return |

---

## Methodology lessons (do not re-learn)

1. Options multi-sleeve risk → **`evaluate_theta_margin` only**.
2. **Exit-day / sum-mode** combiners understate DD (~3% vs ~15% same sleeves).
3. **nav_q** ≠ live; use **fund-mode 2.2× quarterly** for single-account narrative.
4. Literature theta winners cluster in **short RR/skew + put-write**; ATM straddles on VRP/VIX failed bid/ask bar.
5. Intraday edges need **universe + time-of-day + VIX** filters; defaults lose.
6. Short legs / uncapped linear MR / stock Johansen without economic story → OOS blowups.
7. Paper century headlines (industry trend 18%, ORB 42%) ≠ modern ETF/intraday windows.
8. Trade logs are economic truth; merge-step capital scaling is allocation math only.
9. Dip runners need `--yahoo-period max` for 2016+ SMA200 warm-up.
10. Low-ρ satellites (macro AW, TSMOM, Johansen) beat stacking more equity beta.
11. **Ride-rockets:** prefer near-52w-high + concentration blend; do **not** stack near-high + 6m-kill on tight books; S&P PIT beats R3K snapshot for mega-cap rockets; avoid enhanced 7-filter starvation.
11. Intraday MA-slope: **confirm_entry_4b** beats baseline; edge is **positive-skew / tail-driven** (median name≈0%). Prefer **top-5 + max 20% weight** over uncapped top-5. Hard liquidity floors ($10/$20M ADV) kill most return — size caps beat universe cuts.
11. Intraday ORB: **never allow same-bar stop-outs** (entry and stop on one bar understates edge badly). Check stops from `entry_bar+1`; first-touch breakout must not already be through the level on the prior bar.
12. Zarattini ORB quality grid (smoke): **RVOL-rank sizing** helps; **OR range/body filters** and **wider ATR stops** hurt — do not re-propose without new evidence.

---

## Preferred books (pointers only)

Full commands and weights: `BEST_IDEAS.md` · `.cursor/rules/best-ideas-stack.mdc` · `AGENTS.md`.

- **Options Best Ideas (fund-mode):** VRP + lit4 + VXX regime + VXX long call + macro AW + tactical AW + TSMOM + **ride_rockets** + Johansen (± sector / ORB / ma_slope_intraday).
- **Stock-only fund-mode:** Tactical AW + CrackingMarkets dip + TSMOM + Johansen + vol edge + MA slope (top-N + inverse + **intraday**) + QS (± sector).

---

## Session notes (append chronologically)

### 2026-07-18 — Ledger bootstrap

Seeded CSV from transcript synthesis + live industry-trend replication:

- French 48 Timing Industry (1926–2024): CAGR ~18.4%, vol 12.6%, Sharpe ~1.16, DD −33% — **matches paper shape**; status `optional` / `paper_era`.
- Same model on 31 SPDR ETFs (2005–2024): CAGR ~7.8% vs SPY ~10.1%, DD −23% vs −55% — **risk control, not SPY alpha**; status `weak` for raw return, useful as diversifier.
- Apples-to-apples French 48 on 2005–2024: ~9.4% CAGR — most of “ETF underperforms 18%” is **era**, not ETF brokenness.

### 2026-07-18 — Deep transcript / catalog expansion (+93 rows → ~140 total)

Mined QS PDF survey ([4cec0e6e](4cec0e6e-c5fd-42fd-8f5f-c3e4ad0e8a2b)), 100-options brainstorm ([6d02f49d](6d02f49d-de42-4438-a9b1-e0bfc953f007)), Chan PDF ([df5fbac8](df5fbac8-48a0-4c69-9eb1-24829f1ade03)), overnight VIX ([736dfeff](736dfeff-6f92-4d02-a984-ce5bae4435ac)), day-trading docs.

**Backtested additions**
- All **50** QS systematic sleeves from `qs_systematic_rank50_ranking.csv` (hard fails: `S47_gap_up_fade` Sh −4.15, `S06_opex_friday`, `S30_connors_3up_fade`).
- QS composites: `qs_actionable_7`, `qs_top15_book_fit`.
- Full **Chan suite** (best: `ewa_ewc_kalman` Sh 0.96; fails: linear MR, GLD/GDX Bollinger, short-on-gap, TSMOM TLT).
- **Top-9 D MTM** (+81%, Sh 0.48, DD −15%) + D039/D018/D095… passers.
- **SPY overnight × VIX calm/fear** variants (fear nights have higher mean C2C; calm filters weak; oracle = lookahead only).
- Day-trading: `pdl_touch_long` fail, ATR SP100 fail, VWAP reclaim fail, `ma_slope_intraday_confirm4b` promising (vs unconstrained fail).

**Ideation-only catalogs (not tested as that list)**
- QS PDF extras (~35 beyond coded 50)
- 100 unique options brainstorm
- `DAY_TRADING_100_COMPLEMENT_IDEAS.md` (100 ideas)
- Academic delta-hedged VRP / XS IV–HV (never run)

**Note:** The “~200 trade ideas from a PDF” memory maps to the **QuantifiedStrategies survey** chat — we coded/backtested a **rank-50** systematic subset; the rest stays `ideation_only`.

### 2026-07-18 — Granular deep mine (pass 1 → ~824, pass 2 → ~1,796)

**Pass 1:** per-SID catalogs (D000–D099, lit v4/OOS/rank, skew, VXX sweeps, ORB, MA slope tops, French discovery, calendar floor).

**Pass 2:** full grids / secondary screens that were only summarized before:

| Block | ~Rows added | Artifact |
|-------|-------------|----------|
| French low-DD opt **full** | 265 | `french_low_dd_opt_all.csv` |
| French vol-target grid | 103 | `french_vol_target_opt_grid.csv` |
| Markov VIX futures L/S (top/bot + validated) | ~65 | `markov_vix_futures_ls_opt_*.csv` |
| Best Ideas router consistency | 50 | `best_ideas_router_gridsearch_consistency_2022_2025_dd15.csv` |
| AW option matrix + macro complement | 112 | `all_weather_option_matrix_*`, `macro_option_complement_ranked.csv` |
| Lit yearly consistency + base + v3 | 300 | `lit_catalog_yearly_consistency_*`, `literature_search_agent_batch*.json` |
| MA slope inverse SH full | 37 | `ma_slope_inverse_spy_sweep_full.csv` |
| Ride-the-rockets variants | 10 | `ride_rockets_variant_sweep.json` |
| CM intraday ATR/dip tweaks | ~21 | `cm_intraday_*_meta.json` |
| VXX sweet-spot put tops/bots | 25 | `vxx_sweet_spot_put_option_sweep.csv` |

**Pass 3:** French CAGR4 filter (153), router 2022–25 / 2023–24 tops+bots (80), MA-slope stage4 extremes (60), Alpaca intraday stop tops (20), vol-target recommended curve parent → **~2,053 total rows**.

**Catalog pass-rates (still sparse):** D-catalog ~9/100; lit OOS 2022–24 ~3/100; French discovery ~2/83 promising. Full low-DD French overlays populate many `optional`/`promising` low-vol rows — treat as factor research, not live sleeves.

```bash
.venv/bin/python RenTech/strategy_stack/research_ledger_cli.py summary
.venv/bin/python RenTech/strategy_stack/research_ledger_cli.py search markov
.venv/bin/python RenTech/strategy_stack/research_ledger_cli.py list --family options_macro --status promising
```

### 2026-07-18 — Trade-level mine (`tl_*` → ~2,177 rows)

Aggregated **closed-trade logs** into ledger ids `tl_*` + companion `research_ledger_trade_stats.csv` (n, Σpnl, WR, max win/loss, exit-step Sharpe/DD).

| Source | Grain |
|--------|--------|
| VRP bundle JSONL | all + regime + exit_reason |
| lit4+VRP / Top9-D margin CSVs | sid / trade_kind |
| Lit low-corr + legacy4 trade logs | sid / family |
| Multi-sleeve + Sharpe-opt ALL_SLEEVES | sleeve raw + scaled |
| Best Ideas fund ALL_TRADES | sleeve + fund_book |
| IV / VXX engine JSONLs, skew `*_trades.csv` | per file |
| ORB / CM dip / Nasdaq MA | streamed; ORB = **pnl_pct** |

**Grounding checks:** VRP Σpnl ≈ **$98.5k** / 2027 trades / WR 81.5% matches AGENTS baseline. VXX long-call tagged `research_only` (one ~$141k winner). Stop-loss exits: −$41k. Sector ticker-split rows in fund export have $0 pnl (aggregate sleeve holds PnL).

```bash
.venv/bin/python RenTech/strategy_stack/research_ledger_cli.py trades vrp
.venv/bin/python RenTech/strategy_stack/research_ledger_cli.py trades --worst --limit 20
.venv/bin/python RenTech/strategy_stack/research_ledger_cli.py search tl_litvrp
```

### 2026-07-18 — SPY overnight VIX calm vs fear

- Runner: `run_spy_overnight_vix_calm.py` (close→close; features through close ``t``).
- Window 2016-01-04→2026-06-18, $100k. Oracle top-10% nights mean **+191 bps** (lookahead ceiling).
- Feature lift: top nights = **high VIX/VVIX/RSI**, weak SPY — **not calm**.
- `calm_hard_rules`: +37.9%, Sharpe 0.64, DD −7.2%, invested 27%; mean overnight **below** always-in → status `weak`.
- `fear_hard_rules`: +63.2%, Sharpe 0.44, DD −23.3%; mean **+12 bps** vs +6 always; ~24% precision vs oracle → `weak` (edge exists, risk not better than always-in Sharpe 0.89).
- Prefer existing QS overnight sleeves (3-down / 10d-low) for book; do not promote calm-as-top-decile narrative.

### 2026-07-18 — SPY overnight 25-idea grid

- Runner: `run_spy_overnight_ideas25.py` → `spy_overnight_ideas25_ranked.csv`.
- Always overnight: +341% / Sharpe 0.89 / DD −34%.
- **Beat always Sharpe (5):** I18 Turnaround Tue (0.94, DD −4%), I08 3-down (0.93), I21 vol-target 10% (1.04), I15 contango (0.96), I22 skip |ΔVIX|≥20% (0.90).
- **Best selective edge:** I10 fear∩3down (+41 bps/night), I06 VIX RSI≥70 (+35 bps), I08 (+25 bps).
- **Fails:** I14 VIX cooling (−0.28 Sharpe), I16 backwardation fade, I12 fear top15%, C2O fear (I03) worse than C2C.
- Status `research_only` (multiple comparisons / in-sample).

### 2026-07-18 — SPY overnight EQ ranks 1+2+3+5 (**LEDGERIT**)

- Equal-weight mean of sleeve returns: **I18 + I10 + I08 + I09** (skip vol-target I21).
- Window 2016-01-04→2026-06-18, $100k: **+82.2%** · Sharpe **1.05** · max DD **−6.9%** · end ~$182k.
- Coverage: ~20% days with ≥1 sleeve; avg 0.31 sleeves on. I10↔I08 corr **0.90** (limited diversify within pullback cluster; I18 helps).
- Artifacts: `spy_overnight_eq_r1235_{daily,meta}.*`
- Decision: **`promising`** (better Sharpe/DD than always-overnight; not yet in stock book — overlaps QS actionable-4).
- CSV id: `stk_spy_on_eq_r1235`.

### 2026-07-18 — Trigger word **LEDGERIT**

- Always-on rule `.cursor/rules/research-ledger.mdc` now keys off **`LEDGERIT`** (alias `/ledger`).
- Any chat: user says `LEDGERIT` → agent must flush tested ideas/metrics into `research_ledger.csv` + optional session note before ending the turn.
- Auto-append still required after canonical/long backtests without being asked.

### 2026-07-18 — Agentic improve loops (Horizon-style) — lit options + QS stocks

Built and ran generate→evaluate→select→memory→OOS-verify on two spines:

**Options (Theta SPY 15:45 + Yahoo panel)** — `agentic_strategy_loop.py`
- IS 2016–2021 / OOS 2022–2024; param mutate of lit catalog; composite fitness; fingerprint bans.
- **14 promoted, all putwrite.** Top: S055 OOS Sh **1.16**, +2.8%, DD **−0.7%** (1-lot / exit-day).
- Non-putwrite preferred families failed OOS floors → ledger `fail` `agentic_lit_non_putwrite_oos_fail`.
- CSV: `agentic_lit_loop_2016_2024`, `agentic_lit_s055_oos_2022_2024`. Artifacts: `data/logs/agentic_loop/lit_2016_2024/`.

**Stocks (Yahoo QS S01–S50)** — `agentic_stock_strategy_loop.py`
- Same IS/OOS; mutate = EW ID combos; rank by ID.
- **31 promoted.** OOS singles: S42 +74%/Sh 2.99; S45 +61%/Sh 3.00; S44 +99%/Sh 1.86 (DD −10%); best combo **S42+S45** +68%/Sh 3.19/DD −2.0%.
- CSV: `agentic_stock_loop_2016_2024`, `agentic_stock_s42_oos_2022_2024`, `agentic_stock_s45_oos_2022_2024`, `agentic_stock_combo_s42_s45_oos`, `agentic_stock_s44_oos_2022_2024`.
- Artifacts: `data/logs/agentic_stock_loop/qs_2016_2024/`. Caveat `no_costs,yahoo`.

### 2026-07-18 — Zarattini 5m ORB (**LEDGERIT**)

- Paper: SSRN 4729284. Engine: `orb_zarattini.py` / `run_orb_zarattini.py`. Alpaca RTH 1m from **2020** (not 2016).
- **Bug:** same-bar stop-outs → deprecated `stk_orb_5m` (+82% / Sharpe 0.86). Post-fix full-universe OR=5m: **+20,947%** · CAGR **297%** · Sharpe **4.43** · DD **−25.7%** (`orb_or5_2020_2023`, `research_only` / capacity).
- **OR sweep:** 5m ≫ 10m ≫ 15m ≫ 30m; **60m fail** (already ledgered).
- **Quality smoke (400 alpha symbols):** best = **`baseline_rvol_size`** (Sharpe 2.38, +1359%, DD −17.6%) vs baseline 2.22 / +738%. Early-entry cutoff small lift. **Fail:** OR quality filters, 15% ATR stops. **Weak:** top5 RVOL≥3× (high ret, worse DD).
- **Best Ideas:** `--with-orb-zarattini` fund-mode 2.2× → +687% · Sharpe 1.97 · DD −9.0% · ORB wt **6.1%** (`bi_fund_orb_zarattini`, `optional`).
- **Pending:** full-universe confirm of `baseline_rvol_size` (aborted twice; focused 2-preset restart in flight). Prefer promoting that CSV if it wins.

### 2026-07-18 — MA slope intraday confirm_4b + de-tail (**LEDGERIT**)

Chat [MA slope intraday](239aefd9-be9c-4a37-a7a5-dbc75ed32a55): Alpaca 5m day-trade (~500 names).

**What worked**
- `confirm_entry_4b` (rank top-10 @ ~10:25, re-rank +4 bars, enter intersection): **+1,159%** / Sh **1.32** / DD **−31%** vs baseline +509% / 1.09 / −35% (2020→2026-06).
- **De-tail for book:** confirm_4b + **top-5** + **max 20% weight** → **+1,264%** / Sh **1.48** / DD **−32%**. Yearly all positive (2020 +42 … 2024 +99 … 2026 YTD +52).
- Token `ma_slope_intraday` → Best Ideas (`--with-ma-slope-intraday`) + **default on `--stock-only`** (~6.1% wt). Runner: `run_ma_slope_intraday_standard.py`.

**Trade structure (not one winner)**
- Session WR ~53%; name WR ~49.5%; **median name ≈ 0%**; skew ~4.8 → positive-skew / tail-driven.
- Cap each name at +10% stock return → ~+118% (top-10) / ~+234% (top-5). Uncapped top-5 **+8,706%** / DD **−56%** = research_only.
- Rank-gap 5%: modest +98pp (promising add-on). Lag-5: slight return give-up, better DD.

**What failed / hurts**
- VWAP on confirm_4b (−734pp), rising-price confirm, liquidity $10/$20M ADV (+26% only).
- 5bps entry slippage: confirm_4b → +459% / Sh 0.95 (still positive).

**CSV ids:** `ma_slope_intraday_confirm4b_top5_cap20` (default_book), `…_top10`, `…_top5_uncapped` (research_only), `…_baseline`, `…_liq10_adv20m` (fail), `…_vwap` (fail), `…_rising` (fail), `…_rank_gap5pct` (promising), `…_5bps` (weak).

### 2026-07-18 — Ride-the-rockets SPMO session (**LEDGERIT**)

Goal: ride NVDA/AVGO-like relative winners until momentum fades (not SPMO quintile indexer).

**What worked**
- **`ride_rockets` 50/50** = near_52w_high top25 + ten_rockets top10: **+282%** / Sh **0.75** / DD **−21%** / β 0.80; sleeve corr **0.69**. → **Best Ideas** `--with-ride-rockets` (~6% fund wt). Runner: `run_ride_rockets_5050_standard.py`.
- Near-high alone (top25): best Sharpe/DD among 10 variants (**+226%** / 0.74 / **−22.5%**).
- Ten rockets (top10): best absolute return (**+309%** / 0.66 / −29%).
- S&P PIT >> Russell 3000 snapshot for this thesis (R3K quintile +204% / 0.63 / −35%).

**What failed / hurts**
- Stacking near-high + 6m-kill on tight books (**champ**) over-filters (top12 both gates +167% / 0.59).
- Enhanced all-7 filters starve exposure (~20% gross); cash-yield Sharpe artifact.
- Acceleration (80% 6m), raw price rank, sticky buffer — all underperform.
- Legacy bob50 (`res_spmo`) → **deprecated** (low-gross / cash artifact); prefer ride_rockets 50/50.

**CSV ids:** `stk_ride_rockets_5050` (default_book), `stk_ride_near_52w_high_top25` / `stk_ride_ten_rockets_top10` (promising), `stk_ride_champ_stack_overfilter` (weak), `stk_spmo_pit_quintile` (research_only), `stk_spmo_enhanced7_starve` / `stk_ride_accel_raw_sticky_fail` (fail), `stk_r3k_momentum_quintile` (weak), `res_spmo` (deprecated).

### Template for new notes

```
### YYYY-MM-DD — short title
- What was tested (runner + flags)
- Headline metrics + caveat tag
- Decision: promote / keep optional / fail / deprecate
- CSV `id` updated or added
```