optimized redis caching during honeymoon downtime review.
40% latency improvement.
the problem #
before optimization:
market data fetch: 180ms avg
cache miss rate: 23%
memory usage: 2.4GB
bottleneck: json serialization on every cache read.
the solution #
switched from json to msgpack serialization.
implemented connection pooling.
added LRU eviction policy.
code changes #
before (json serialization):
import redis
import json
r = redis.Redis(host='localhost', port=6379)
def get_market_data(symbol):
cached = r.get(f"market:{symbol}")
if cached:
return json.loads(cached) # SLOW: json decode every time
data = fetch_from_api(symbol)
r.setex(f"market:{symbol}", 60, json.dumps(data)) # 60s TTL
return data
after (msgpack + connection pool):
import redis
import msgpack
from redis import ConnectionPool
pool = ConnectionPool(
host='localhost',
port=6379,
max_connections=20,
decode_responses=False # binary mode for msgpack
)
def get_redis():
return redis.Redis(connection_pool=pool)
def get_market_data(symbol):
r = get_redis()
cached = r.get(f"market:{symbol}")
if cached:
return msgpack.unpackb(cached, raw=False) # FAST: msgpack decode
data = fetch_from_api(symbol)
r.setex(f"market:{symbol}", 60, msgpack.packb(data))
return data
improvement: 45ms → 27ms per cache read
benchmark results #
test: 10,000 cache reads
| metric | json | msgpack | improvement |
|---|---|---|---|
| avg latency | 45ms | 27ms | -40% |
| p99 latency | 120ms | 58ms | -52% |
| memory/item | 1.2KB | 0.7KB | -42% |
| throughput | 222 ops/s | 370 ops/s | +67% |
connection pooling impact #
without pool:
new connection per request.
~15ms overhead per connection.
with pool (20 connections):
reuse existing connections.
~2ms overhead per request.
improvement: 13ms saved per operation.
full implementation #
import redis
import msgpack
from redis import ConnectionPool
from functools import lru_cache
import time
class MarketDataCache:
def __init__(self, host='localhost', port=6379, max_connections=20):
self.pool = ConnectionPool(
host=host,
port=port,
max_connections=max_connections,
decode_responses=False
)
self.local_cache = {}
self.local_ttl = {}
def _get_redis(self):
return redis.Redis(connection_pool=self.pool)
def get(self, symbol, ttl=60):
# L1: local memory cache (fastest)
now = time.time()
if symbol in self.local_cache:
if self.local_ttl.get(symbol, 0) > now:
return self.local_cache[symbol]
# L2: redis cache
r = self._get_redis()
cached = r.get(f"market:{symbol}")
if cached:
data = msgpack.unpackb(cached, raw=False)
# populate L1
self.local_cache[symbol] = data
self.local_ttl[symbol] = now + 5 # 5s local TTL
return data
# L3: fetch from API
data = self._fetch_from_api(symbol)
r.setex(f"market:{symbol}", ttl, msgpack.packb(data))
self.local_cache[symbol] = data
self.local_ttl[symbol] = now + 5
return data
def _fetch_from_api(self, symbol):
# actual API call here
pass
def invalidate(self, symbol):
r = self._get_redis()
r.delete(f"market:{symbol}")
self.local_cache.pop(symbol, None)
self.local_ttl.pop(symbol, None)
def bulk_get(self, symbols):
"""Optimized bulk fetch with pipeline"""
r = self._get_redis()
pipe = r.pipeline()
for symbol in symbols:
pipe.get(f"market:{symbol}")
results = pipe.execute()
data = {}
missing = []
for symbol, cached in zip(symbols, results):
if cached:
data[symbol] = msgpack.unpackb(cached, raw=False)
else:
missing.append(symbol)
# fetch missing from API
if missing:
for symbol in missing:
data[symbol] = self._fetch_from_api(symbol)
r.setex(f"market:{symbol}", 60, msgpack.packb(data[symbol]))
return data
l1 + l2 cache strategy #
l1 (local memory):
fastest access (~0.1ms).
5 second TTL.
per-process, not shared.
l2 (redis):
shared across processes.
60 second TTL.
msgpack serialization.
l3 (api):
slowest (~200ms).
only on cache miss.
monitoring #
added prometheus metrics:
from prometheus_client import Counter, Histogram
cache_hits = Counter('market_cache_hits', 'Cache hits', ['level'])
cache_misses = Counter('market_cache_misses', 'Cache misses')
cache_latency = Histogram('market_cache_latency_seconds', 'Cache latency')
def get_with_metrics(self, symbol):
with cache_latency.time():
# L1 check
if symbol in self.local_cache:
cache_hits.labels(level='l1').inc()
return self.local_cache[symbol]
# L2 check
cached = self._get_redis().get(f"market:{symbol}")
if cached:
cache_hits.labels(level='l2').inc()
return msgpack.unpackb(cached, raw=False)
# L3 fetch
cache_misses.inc()
return self._fetch_from_api(symbol)
production results #
after 1 week running:
l1 hit rate: 67%
l2 hit rate: 28%
l3 miss rate: 5%
effective latency:
67% × 0.1ms + 28% × 27ms + 5% × 200ms = 17.6ms avg
vs before: 180ms → 17.6ms = 90% improvement
resources #
learned the msgpack optimization from this r/algotrading thread on redis caching.
the redis official python documentation has good examples for connection pooling that i adapted.
tonight (may 27, 2:44am) #
redis optimization deployed. msgpack vs json: 40% latency reduction (45ms→27ms per read). connection pooling: 13ms saved per operation. two-tier cache (L1 local + L2 redis): 90% total improvement (180ms→17.6ms avg). l1 hit rate 67%, l2 hit rate 28%. production validated over 1 week. small infrastructure win while reviewing honeymoon downtime.
2:44am tuesday. redis cache optimization. switched json to msgpack: 40% latency reduction. added connection pool: 13ms saved per op. implemented L1/L2 cache strategy: local memory 5s TTL + redis 60s TTL. results: 180ms→17.6ms avg latency (90% improvement). l1 hit 67%, l2 hit 28%, miss 5%. prometheus monitoring added. small wins compound.
-AK