ZilLdZddlmZddlmZddlmZddlZddl Z ddl Z dZ edZddZdd dd ZeGd d Zed k(rGeeZej-ddZeej3edeeyy#e$rZdZ eZ YdZ[zdZ[wwxYw)a Historical market data loading and normalization for a multi-timeframe system. - Daily bars: macro momentum filter (e.g. SPY/QQQ regime). - Intraday bars (1h / 15m): micro mean-reversion / stat-arb style signals. Dependencies: pandas, numpy, yfinance (install: pip install yfinance). ) annotations) dataclass)LiteralN)15m30m60m1h1dc4ttdty)NzSyfinance is required. Install with: python -m pip install yfinance Original error: )yf ImportError_YF_IMPORT_ERROR>/opt/rentech/trading_bot/RenTech/strategy_stack/data_loader.py_require_yfinancers) z /0 2  rT)prefer_adjustedc|jrtjS|j}t |j tj rm|j jdkDrT|j jdk(r|j jdn|j jd|_dddddd d }|j|jDcic]\}}||j vs||c}} }|rd|j vr|d|d<gd }|Dcgc]}||j vs|}}|r$td |dt|j ||jtj }|j#}||j$j'd}|Scc}}wcc}w)z Normalize yfinance output to lowercase OHLCV columns with a DatetimeIndex. Handles both single-ticker (flat columns) and occasional MultiIndex columns. ropenhighlowclose adj_closevolume)OpenHighLowClosez Adj CloseVolume)columns)rrrrrz0Expected OHLCV columns missing after normalize: z. Got: last)keep)emptypd DataFramecopy isinstancer" MultiIndexnlevels droplevelget_level_valuesrenameitems ValueErrorlistastypenpfloat64 sort_indexindex duplicated) dfrout rename_mapkvrequiredcmissings r_standardize_ohlcvr@&s  xx||~ '')C#++r}}- ;;   "69kk6I6IQ6N#++//2TWT_T_TpTpqrTsCK  J **z/?/?/AVtq!Q#++EUadV* WC;#++5;'G 9H";Qas{{&:q;G;KG9T[\`adalal\m[nopp h-  rzz *C .. C syy###00 1C JW >> loader = DataLoader() >>> daily = loader.fetch_daily("SPY", period="5y") >>> hourly = loader.fetch_intraday("AAPL", interval="1h", period="60d") Fbool auto_adjustN)startendperiodc@ttj|}|r|j|d|j}n|j||d|j}t |}t j|jjd|_|S)z Load daily OHLCV for momentum / regime models. Provide either (start & end) or period (e.g. '5y', 'max'), not both if yfinance conflicts. r rGintervalrDrErFrJrDN) rr TickerhistoryrDr@r& to_datetimer6 tz_localize)selftickerrErFrGtrawr9s r fetch_dailyzDataLoader.fetch_daily]s  IIf  ))6DdFVFV)WC))%S4TM]M])^C %NN399-99$?  rr )rJrErFrGct|dvrdn|}tj|}|r|j|||j}n|j||||j}t |} t j| j| _| jj/| jjdjd| _| S)z Load intraday OHLCV for mean-reversion / rolling z-score engines. Note: Yahoo limits intraday depth (often ~730 days for hourly; shorter for 15m). Prefer `period='60d'` or similar when using 15m bars. )r rrrIrKNUTC) rr rLrMrDr@r&rNr6tz tz_convertrO) rPrQrJrErFrGivrRrSr9s rfetch_intradayzDataLoader.fetch_intradayts -/UX IIf  ))6BDDTDT)UC))%S24K[K[)\C %NN399- 99<< # ,,U3??ECI rzAmerica/New_York)day_tzcB|jr|S|j}|r@|j|jdn|j d}|j |}n|}|j }t j|jj|d<|S)z Optional: assign a calendar date column for merging intraday bars with daily regime. Intraday index is timestamp; we add `trade_date` as the session date in `day_tz`. rV trade_date) r%r6rWrOrXr(r&rN normalizedate)rPintradayr[idxtsr9s ralign_to_trading_daysz DataLoader.align_to_trading_dayss >>Onn +.66>'s~~e?TBv&BBmmoNN2<<>+>+>?L r) rQstrrEstr | pd.Timestamp | NonerFrerG str | Nonereturn pd.DataFrame) rQrdrJIntradayIntervalrErerFrerGrfrgrh)r`rhr[rfrgrh) __name__ __module__ __qualname____doc__rD__annotations__rTrZrcrrrrBrBNsKi ,0)-! )  '    6&*+/)-!#  )  '  B0    rrB__main__SPY6mo)rGrows)rgNone)r8rhrrCrgrh)rm __future__r dataclassesrtypingrnumpyr3pandasr&yfinancer rr erirr@rBrjdlrTdprinttaillenrrrrs#! :; EI%P WW Wt z B uU+A !&&(O &#a& k BsBB#BB#