Redis

2:30 AM monday. week one of what i’m calling “the post-tariff-chaos era” starts in a few hours. last week was one of those that splits into a clear before and after. monday and tuesday felt like freefall — VIX went from 20 to 32 in about 36 hours, SPX dropped hard, options spreads blew out 3-4x, and my event risk throttle (which I built the week prior and wrote about here) was earning every line of code it took to build. then wednesday happened. whoever made the tariff pause call did it at 1:07 PM eastern and watching the S&P rip 8% in ninety minutes while running algorithms was… a lot.

2:15 AM friday. couldn’t sleep after the week we just had. VIX ripped to 28 monday, calmed down midweek, then did that whipsaw thing thursday afternoon where you think it’s done but it’s absolutely not done.

2:30 AM wednesday. A. left the kitchen light on when she went to bed, which means she had a late session too. checked on her around midnight — still at her desk, headphones on, coding something for a client. now she’s asleep and I’m at mine.

2:30 AM monday. Q2 week 2 starts in a few hours. Been sitting with something since Thursday when I posted the Q2 week 1 numbers. Said we were running at 60% position size - waiting for the health scoring system to validate before going full deployment.

2:15am wednesday. still processing this week. the q1 factor attribution post from sunday was cathartic but it also made me confront something i’d been papering over: i was flying blind on real-time greeks for most of march. not completely blind — i had position-level greeks from IB’s TWS feed. but aggregating them into a coherent portfolio view? that was a manual spreadsheet thing i’d run every few hours.

lied about sleeping. got into bed, laid there for 45 minutes, kept thinking about something. went back to the desk. the signal decay issue i diagnosed tonight (latency routing on crypto momentum) is real and i fixed it. but while i was digging through three months of fill data, i noticed something else. something i’d been ignoring entirely.

moved all my real-time market data from postgres to redis about 8 weeks ago. latency dropped from ~15ms to sub-millisecond. should’ve done this way earlier. the postgres problem # when i first built my algo infrastructure i stored everything in postgres because that’s what i knew. tick data, order book snapshots, greeks, everything went into timescaledb (postgres extension for time-series).

just finished a redis optimization project. latency went from 45ms to 8ms. here’s how. the problem # market data pipeline was bottlenecking at redis.

optimized redis caching during honeymoon downtime review. 40% latency improvement. the problem # before optimization: market data fetch: 180ms avg

real-time data = critical for algo trading. redis = in-memory cache for speed. python pipeline implementation. the latency problem # pulling data every request: