2:15 AM wednesday. apartment quiet. A. went to bed around midnight — she had a client deadline today so it was a long one. checked the colo heartbeat before sitting down to write this. normal. algos running clean for the first time since last monday.
been staring at the tariff week data for two days. monday’s postmortem surfaced something i’ve been dancing around for a while: my algos don’t know how confident they should be. they know when not to trade — VIX threshold, event windows, my throttle layer. but they don’t have a real-time read on how much to trust their own signals moment-to-moment.
that’s the gap. this post is about closing it.
the gap #
last week my IV rank signal was “working correctly.” it computed the right number from the right data. IV rank was genuinely elevated at 82%, which theoretically signals better premium selling conditions. the logic was sound.
except the market was in freefall, spreads on SPX options were 3-4x their normal width, the data feeding that IV rank was 12 minutes stale by the time each cycle ran, and three of my sub-signals were pointing in conflicting directions. so “IV rank is high, favorable for premium selling” was technically true and completely useless at the same time.
no signal flagged itself as unreliable. the event risk throttle blocked new entries correctly — that was the hard binary failsafe. but what i wanted was something softer: a continuous real-time read on how reliable the signal environment is, and a proportional response to that read.
i’m calling it the Signal Quality Score (SQS). composite 0-100 per strategy. four weighted components. below threshold → system doesn’t open new positions. runs async, publishes to Redis, stores history in TimescaleDB.
built it last weekend. here’s how it works.
SQS: four components, one number #
data freshness (30% weight): how stale is the input data? IV rank from 12 minutes ago during a vol spike is a different instrument than IV rank from 45 seconds ago. linear decay from 100 → 0 over a 5-minute window. the tariff week problem directly: 15-minute recalculation cadence was already hitting staleness penalties before the bad entries happened.
regime deviation (35% weight): how far is current VIX from the rolling baseline? uses z-score off recent VIX history. small deviations get a light penalty. extreme moves — VIX jumping 60% in 36 hours like last week — crush the score exponentially. IV rank > 75% applies an additional penalty. the model trained on “normal” conditions loses edge when operating outside that distribution. this component quantifies that.
spread quality (20% weight): bid-ask spread as a multiple of the symbol’s 25th-percentile historical baseline. 1x baseline → 100. 3x baseline → ~25. 5x → ~0. SPX spreads went 3-4x normal monday-tuesday. this component was 20-30 during the worst of it.
signal agreement (15% weight): directional agreement across sub-signals. IV rank bullish, VIX term structure neutral, momentum bearish — those three are pointing different ways. high standard deviation across the signal votes → lower score. catches the case where each individual signal is technically valid but they’re collectively incoherent.
composite = weighted average. each strategy has a threshold. below threshold → no new positions.
SQS through tariff week for the SPX premium selling strategy:
threshold is 65. SPX SQS dropped to 35 by wednesday morning april 8 — almost 30 points below the trade gate. system had no new SPX entries from monday afternoon through friday morning. that’s the correct call. crypto SQS stayed above its 50 threshold most of the week because momentum signals hold more coherently through equity vol spikes.
the code #
import asyncio
import time
from dataclasses import dataclass
from typing import Dict, Optional
from collections import deque
import numpy as np
import redis.asyncio as redis
@dataclass
class ComponentScore:
score: float # 0-100
weight: float
name: str
reason: str
timestamp: int # unix ms
@dataclass
class SignalQualityScore:
strategy_id: str
composite: float # 0-100 weighted composite
components: Dict[str, ComponentScore]
timestamp: int
trade_permitted: bool
threshold: float
class SignalQualityScorer:
def __init__(
self,
redis_client: redis.Redis,
strategy_thresholds: Dict[str, float],
baseline_vix: float = 16.5,
max_staleness_ms: int = 300_000, # 5 minutes
):
self.redis = redis_client
self.thresholds = strategy_thresholds
self.baseline_vix = baseline_vix
self.max_staleness_ms = max_staleness_ms
self._vix_history: deque = deque(maxlen=100)
self._spread_history: Dict[str, deque] = {}
async def score_data_freshness(
self,
data_ts_ms: int,
now_ms: Optional[int] = None,
) -> ComponentScore:
"""Linear decay: fresh = 100, beyond max_staleness = 0."""
now = now_ms or int(time.time() * 1000)
age_ms = now - data_ts_ms
if age_ms <= 0:
score, reason = 100.0, "data is live"
elif age_ms >= self.max_staleness_ms:
score = 0.0
reason = f"data {age_ms/1000:.0f}s old — exceeds {self.max_staleness_ms/1000:.0f}s limit"
else:
score = 100.0 * (1.0 - age_ms / self.max_staleness_ms)
reason = f"data {age_ms/1000:.1f}s old"
return ComponentScore(score=score, weight=0.30, name="data_freshness",
reason=reason, timestamp=now)
async def score_regime_deviation(
self,
current_vix: float,
iv_rank: Optional[float] = None,
) -> ComponentScore:
"""
Exponential penalty for vol regime distance from rolling baseline.
Models calibrated on normal conditions degrade in extreme regimes.
Small deviations get light penalty. Extreme moves get crushed.
"""
self._vix_history.append(current_vix)
now = int(time.time() * 1000)
if len(self._vix_history) < 10:
deviation = abs(current_vix - self.baseline_vix) / self.baseline_vix
else:
hist = np.array(self._vix_history)[:-1]
z = abs(current_vix - np.mean(hist)) / (np.std(hist) + 1e-6)
deviation = z / 3.0 # z=3 → full deviation (score ≈ 0)
# exp(-2.5 * deviation): at deviation=0 → 100, at deviation=1 → ~8
score = float(np.clip(100.0 * np.exp(-2.5 * deviation), 0.0, 100.0))
reason = f"VIX={current_vix:.1f}, regime_deviation={deviation:.2f}"
if iv_rank is not None and iv_rank > 75:
iv_penalty = (iv_rank - 75) / 25 * 20.0 # up to -20 at IVR=100
score = max(0.0, score - iv_penalty)
reason += f", IVR={iv_rank:.0f}% → penalty applied"
return ComponentScore(score=score, weight=0.35, name="regime_deviation",
reason=reason, timestamp=now)
async def score_spread_quality(
self,
symbol: str,
bid_ask_pct: float, # spread as fraction of mid price
) -> ComponentScore:
"""
Tracks spread vs symbol's 25th-percentile historical baseline.
3x baseline → ~25. 5x → ~6. Widening = worse execution environment.
"""
now = int(time.time() * 1000)
if symbol not in self._spread_history:
self._spread_history[symbol] = deque(maxlen=200)
hist = self._spread_history[symbol]
hist.append(bid_ask_pct)
if len(hist) < 20:
score = float(np.clip(100.0 - bid_ask_pct * 80.0, 0.0, 100.0))
reason = f"spread={bid_ask_pct:.4f} (static baseline, need more history)"
else:
baseline = float(np.percentile(list(hist)[:-1], 25))
ratio = bid_ask_pct / (baseline + 1e-6)
# ratio=1 → 100, ratio=3 → ~25, ratio=5 → ~6
score = float(np.clip(100.0 / (1.0 + (ratio - 1.0) * 2.0), 0.0, 100.0))
reason = f"spread={bid_ask_pct:.4f}, {ratio:.1f}x baseline ({baseline:.4f})"
return ComponentScore(score=score, weight=0.20, name="spread_quality",
reason=reason, timestamp=now)
async def score_signal_agreement(
self,
signal_votes: Dict[str, float], # name → directional confidence [-1, 1]
) -> ComponentScore:
"""
High variance across sub-signal directions = incoherent environment.
Agreement and conviction both matter. Agreement weighted 60/40.
signal_votes: {'iv_rank': 0.7, 'vix_term': 0.3, 'momentum': -0.4}
"""
now = int(time.time() * 1000)
if not signal_votes:
return ComponentScore(score=50.0, weight=0.15, name="signal_agreement",
reason="no votes", timestamp=now)
vals = np.array(list(signal_votes.values()))
agreement = float(100.0 * np.exp(-3.0 * np.std(vals))) # low std → high agreement
conviction = float(100.0 * abs(np.mean(vals))) # strong direction → higher
score = float(np.clip(0.6 * agreement + 0.4 * conviction, 0.0, 100.0))
reason = (f"n={len(vals)}, mean={np.mean(vals):.2f}, "
f"std={np.std(vals):.2f}")
return ComponentScore(score=score, weight=0.15, name="signal_agreement",
reason=reason, timestamp=now)
async def compute(
self,
strategy_id: str,
data_ts_ms: int,
current_vix: float,
symbol: str,
bid_ask_pct: float,
signal_votes: Dict[str, float],
iv_rank: Optional[float] = None,
) -> SignalQualityScore:
"""
Compute composite SQS and publish to Redis.
Executor polls sqs:{strategy_id} before opening any position.
TTL=60s: if scorer goes silent, key expires → executor blocks automatically.
"""
freshness, regime, spread, agreement = await asyncio.gather(
self.score_data_freshness(data_ts_ms),
self.score_regime_deviation(current_vix, iv_rank),
self.score_spread_quality(symbol, bid_ask_pct),
self.score_signal_agreement(signal_votes),
)
components = {c.name: c for c in [freshness, regime, spread, agreement]}
total_w = sum(c.weight for c in components.values())
composite = round(float(np.clip(
sum(c.score * c.weight for c in components.values()) / total_w,
0.0, 100.0
)), 1)
threshold = self.thresholds.get(strategy_id, 60.0)
trade_permitted = composite >= threshold
# Atomic publish: set key + pub/sub notification
pipe = self.redis.pipeline()
pipe.set(f"sqs:{strategy_id}", str(composite), ex=60)
pipe.publish(f"sqs_update:{strategy_id}", f"{composite}:{int(trade_permitted)}")
await pipe.execute()
return SignalQualityScore(
strategy_id=strategy_id,
composite=composite,
components=components,
timestamp=int(time.time() * 1000),
trade_permitted=trade_permitted,
threshold=threshold,
)
# ── strategy thresholds ────────────────────────────────────────────────────────
STRATEGY_THRESHOLDS = {
"spx_premium_selling": 65.0, # conservative — most sensitive to data quality
"qqq_iron_condors": 65.0,
"es_futures_momentum": 70.0, # highest bar — fastest-moving signals, worst to be stale
"btc_eth_momentum": 50.0, # more robust to equity vol regime shifts
"altcoin_breakout": 45.0, # crypto is inherently chaotic, lower bar
}
async def main():
r = redis.from_url("redis://localhost:6379", decode_responses=True)
scorer = SignalQualityScorer(
redis_client=r,
strategy_thresholds=STRATEGY_THRESHOLDS,
baseline_vix=16.5,
)
# example: SPX strategy, 45 seconds of data staleness, VIX at 21.3
sqs = await scorer.compute(
strategy_id="spx_premium_selling",
data_ts_ms=int(time.time() * 1000) - 45_000,
current_vix=21.3,
symbol="SPX",
bid_ask_pct=0.0018,
signal_votes={
"iv_rank": 0.62, # bullish for premium selling
"vix_term": 0.41, # contango → favorable
"momentum_spy": -0.18, # slight bearish disagreement
},
iv_rank=58.0,
)
print(f"\nstrategy: {sqs.strategy_id}")
print(f"composite SQS: {sqs.composite} | threshold: {sqs.threshold} | permitted: {sqs.trade_permitted}")
for name, comp in sqs.components.items():
print(f" {name:20s} {comp.score:5.1f} ({comp.reason})")
if __name__ == "__main__":
asyncio.run(main())
around 190 lines. the whole complexity lives in the scorer. the executor stays simple.
infrastructure integration #
the executor’s decision loop at the colo now looks like this:
async def can_open_position(strategy_id: str) -> bool:
sqs_raw = await redis_client.get(f"sqs:{strategy_id}")
if sqs_raw is None:
return False # scorer silent or TTL expired — block all entries
return float(sqs_raw) >= STRATEGY_THRESHOLDS[strategy_id]
two lines. the whole complexity lives upstream in the scorer. TTL is the failsafe — if the SQS computation process dies or the chicago box loses connectivity for 60 seconds, the redis keys expire automatically and the executor stops opening positions. no explicit kill switch needed.
timescaledb stores every SQS reading with component-level breakdown. that’s how i built the chart above — dumped april 7-15 history and plotted the degradation curve. the storage schema:
CREATE TABLE sqs_history (
ts TIMESTAMPTZ NOT NULL,
strategy_id TEXT NOT NULL,
composite FLOAT NOT NULL,
freshness FLOAT,
regime FLOAT,
spread FLOAT,
agreement FLOAT,
permitted BOOLEAN
);
SELECT create_hypertable('sqs_history', 'ts');
CREATE INDEX ON sqs_history (strategy_id, ts DESC);
compresses well. cheap to query. one year of history is maybe 400MB for five strategies at 3-minute update intervals.
current status #
where things stand tonight:
all five strategies above their thresholds. VIX is sitting at 21 — still elevated from pre-tariff-week levels (was 16-17 in march) but nowhere near last week’s peak. data freshness improved too: i bumped the IV rank recalculation cadence from 15 minutes to 3 minutes as a separate fix. that alone adds ~15 points to the freshness component under normal conditions.
what would this have changed #
two bad SPX entries on monday april 7, opened in the ~40-minute window between market open and when the event risk throttle fully engaged. SQS was already at ~58 by 10 AM and falling fast. below the 65 threshold. those entries would have been blocked.
those positions held and eventually recovered, but they drew down about 3.2k before they did. SQS would have avoided that.
would have also blocked some entries wednesday afternoon during the tariff pause rally. that’s the other side. the SQS was still low (spread and freshness hadn’t recovered yet even though prices were ripping) so some legitimate opportunity was missed. rough estimate: 1.2k in missed upside.
net: +3.2k avoided drawdown, -1.2k missed upside. about 2k net positive. and the miss-on-the-rally problem is tunable — there’s a smoothed recovery curve i want to test that reweights spread quality lower when VIX is already falling hard. work for this weekend.
thoughts #
the interesting design constraint was keeping SQS directionally unaware. it doesn’t know if you’re long or short, bullish or bearish. it only models the quality of the environment the signals are operating in. actual trade direction is the signal’s job. SQS is the metacognitive layer. they stay separate.
dad was VP of engineering at a biotech. control systems were his thing — he was always talking about feedback loops, closed-loop vs open-loop systems. he’d have called this “finally closing the loop on signal reliability.” would have asked me why it took two years to build. fair question, honestly.
spent some time in the NexusFi VIX and volatility thread over the last few weeks — lots of professional traders in there working through the same regime detection problems from different angles. useful for sanity-checking approaches even if the implementations are completely different.
next: historical calibration. what thresholds would have been optimal across 2023-2025 to maximize blocked-bad-trades while minimizing blocked-good-trades? that’s an optuna optimization job and there’s no clean answer. thresholds probably need to be market-regime-specific too, not just strategy-specific. more work.
for now, system is live. all strategies green. going to sleep.
-AK