Infrastructure

moved execution server to chicago colo march 2024. 3 months data in. latency dropped 67ms → 12ms average. why chicago # CME exchange location: chicago

data feeds = foundation of algo trading. garbage data = garbage trades. i’ve used both polygon.io and alpha vantage extensively. spent months researching data feeds when i started trading. NexusFi community helped narrow down options to these two.

chicago colocation server needed upgrade. 1gbe connection = bottleneck. current setup # location: chicago datacenter (equinix CH1)

been storing market data in regular postgres. works but slow for time-series queries. migrated to timescaledb this week. 10x speedup on historical queries. got the idea from NexusFi algo infrastructure discussions about optimizing market data storage. someone mentioned timescaledb and i researched it.

been running synchronous data fetching since january. works but slow during market open. refactored to async this week. 3x speed improvement. the problem with sync code # # Old synchronous approach def fetch_market_data(symbols): results = [] for symbol in symbols: data = fetch_from_api(symbol) # Blocks here results.append(data) return results # With 10 symbols, takes 10 * 180ms = 1,800ms total each API call blocks until complete.

been noticing market data latency creeping up. average fetch time: 180ms from polygon API. slowing down entry execution. the problem # every time algo needs current price:

A. came over sunday afternoon. gave her full tour of my trading setup. she fucking loved it. the setup tour # server rack:

spent last 3 days rebuilding backtest optimization pipeline. went from 6 hours to 35 minutes for full parameter sweep. the problem # old approach: sequential parameter testing.

rewrote my market data pipeline to use async. 3x faster, way cleaner code. the problem # old synchronous code:

got asked on r/algotrading how i organize my trading repos. here’s my setup after 4 months of refactoring. repo structure # i have 4 main repos: