Skip to main content

strategy health scoring: detecting algo decay before it wrecks your Q2

2:45am monday.

first trading day of Q2. Q1 is officially in the rearview — closed at basically flat, full numbers are in friday’s post. the weekend was heavy. not going into it right now. but Q2 starts regardless, and the algos don’t wait for you to process.

been sitting with this specific thought since the mid-march vol spike: i had solid infrastructure monitoring. prometheus scraping everything, grafana dashboards, circuit breakers that actually fired. the risk engine did its job. what it didn’t do is tell me that my SPX premium selling strategy was quietly losing statistical edge starting about three weeks before the drawdown.

that’s the gap i’ve been thinking about. there’s a difference between monitoring infrastructure health and monitoring strategy health. most traders — even systematic ones — conflate these. they’re not the same thing at all.

infrastructure health: is my server responding, is the data feed live, is the order connection up, are my positions reconciling correctly. this is solved. it’s table stakes in 2026.

strategy health is asking something fundamentally different: is this strategy still doing what it was statistically designed to do? is the signal still predictive? are the parameters still appropriate for current regime? is the edge decaying?

those questions don’t have binary answers, and prometheus can’t scrape them.

the four failure modes
#

before the health score, you need to understand what you’re actually measuring. strategies die in four main ways:

signal decay — the predictive relationship between your indicator and forward returns weakens. this is the most common. edges get arbitraged, market microstructure changes, new participants enter. your strategy keeps firing signals, but those signals are increasingly just noise in expensive disguise.

parameter drift — your calibrated parameters (lookback window, threshold, vol multiplier, etc) were optimal in a past regime but the optimal values have shifted. the strategy isn’t broken, it’s miscalibrated. like running your car with the wrong tire pressure — it still drives, but not well.

regime mismatch — your strategy was designed and tested in a specific market regime (mean-reverting, trending, low-vol, high-vol) and is now operating in the opposite one. SPX premium selling was built for vol normalization and range-bound conditions. Q1 gave us trending vol with persistent skew. structural mismatch.

correlation breakdown — if you run multiple strategies expecting diversification, you need to track cross-strategy correlation in real time. in stressed conditions, strategies that historically had low correlation can suddenly move together. you think you have 4 independent exposures. you actually have 4 trades all long the same factor.

each of these is gradual. none of them announce themselves with a klaxon. they degrade your edge quietly until one bad week turns a manageable drawdown into a painful one.

building the health score
#

composite score, 0-100. four components, each contributing based on how much i trust it for each strategy type.

component 1: rolling information ratio

compare current rolling IR to the strategy’s historical baseline IR. if the rolling 20-session IR drops below 0.3x the baseline, something’s wrong.

import numpy as np
import pandas as pd
from dataclasses import dataclass
from typing import Dict, Tuple, Optional

def rolling_information_ratio(returns: pd.Series, window: int = 20) -> pd.Series:
    """Rolling IR vs zero benchmark."""
    roll = returns.rolling(window)
    ir = roll.mean() / roll.std().replace(0, np.nan)
    return ir.fillna(0)

def ir_health_score(
    current_ir: float,
    baseline_ir: float,
    floor: float = -0.5
) -> float:
    """
    Normalize current IR to 0-100 vs historical baseline.
    baseline_ir: long-run average IR in favorable conditions
    """
    if baseline_ir <= floor:
        return 50.0  # undefined baseline, neutral score
    normalized = (current_ir - floor) / (baseline_ir - floor)
    return float(np.clip(normalized * 100, 0, 100))

component 2: signal predictiveness

rolling pearson correlation between signal value at time T and realized PnL at T+1. if your signal is doing its job, this should be positive and stable. when it flips negative or collapses to noise, edge is gone.

def signal_predictiveness_score(
    signals: pd.Series,
    pnl: pd.Series,
    window: int = 30,
    expected_corr: float = 0.12
) -> pd.Series:
    """
    Rolling correlation between signal and next-period PnL.
    expected_corr: what healthy correlation looks like for this strategy
    """
    forward_pnl = pnl.shift(-1)
    rolling_corr = signals.rolling(window).corr(forward_pnl)

    # Normalize to 0-100
    # expected_corr maps to ~75 (healthy but not exceptional)
    # 0 correlation maps to 40 (marginal)
    # negative correlation maps to 0 (actively broken)
    score = np.clip((rolling_corr / expected_corr) * 75, 0, 100)
    return score.fillna(40)

component 3: parameter stability

i run a fast walk-forward weekly to check if optimal parameters have drifted from calibration. small drift is normal. large drift means the strategy’s assumptions about market dynamics are no longer valid.

@dataclass
class ParameterSpec:
    name: str
    calibrated_value: float
    min_value: float
    max_value: float
    tolerance_pct: float = 0.15  # 15% drift = start watching

def parameter_stability_score(
    current_optimal: Dict[str, float],
    param_specs: Dict[str, ParameterSpec]
) -> float:
    """
    Score based on how far current optimal params drifted from calibration.
    Returns 0-100.
    """
    drift_scores = []
    for param_name, spec in param_specs.items():
        if param_name not in current_optimal:
            continue
        param_range = spec.max_value - spec.min_value
        if param_range == 0:
            continue
        drift = abs(current_optimal[param_name] - spec.calibrated_value) / param_range
        # Scale: 0 drift = 100, tolerance_pct drift = 75, 0.5 drift = 0
        if drift <= spec.tolerance_pct:
            score = 100 - (drift / spec.tolerance_pct) * 25
        else:
            excess = (drift - spec.tolerance_pct) / (0.5 - spec.tolerance_pct)
            score = 75 - np.clip(excess, 0, 1) * 75
        drift_scores.append(score)
    return float(np.mean(drift_scores)) if drift_scores else 50.0

component 4: regime alignment

is the market regime your strategy was designed for matching current conditions? i use a simple 2x2 classifier — trending vs mean-reverting, high vol vs low vol. most strategies are designed for one or two of these quadrants.

class RegimeClassifier:
    def __init__(self, vol_window: int = 20, trend_window: int = 50):
        self.vol_window = vol_window
        self.trend_window = trend_window

    REGIMES = {
        (True, True): 'trend_high_vol',
        (True, False): 'trend_low_vol',
        (False, True): 'mean_rev_high_vol',
        (False, False): 'mean_rev_low_vol',
    }

    def classify(self, price_series: pd.Series) -> str:
        rv = price_series.pct_change().rolling(self.vol_window).std()
        rv_pct = rv.rank(pct=True)
        mom = price_series.pct_change(self.trend_window).abs()
        trend_pct = mom.rank(pct=True)

        is_high_vol = bool(rv_pct.iloc[-1] > 0.6)
        is_trending = bool(trend_pct.iloc[-1] > 0.5)
        return self.REGIMES[(is_trending, is_high_vol)]

    def alignment_score(self, current: str, designed_for: list[str]) -> float:
        if current in designed_for:
            return 100.0
        vol_match = ('high_vol' in current) == any('high_vol' in r for r in designed_for)
        trend_match = ('trend' in current) == any('trend' in r for r in designed_for)
        return 25.0 + (25.0 * int(vol_match)) + (25.0 * int(trend_match))

assembling the composite:

class StrategyHealthMonitor:

    COMPONENT_WEIGHTS = {
        'ir': 0.30,
        'signal': 0.25,
        'params': 0.25,
        'regime': 0.20,
    }

    HEALTH_BANDS = [
        (70, 'healthy',   'run at full allocation'),
        (50, 'degraded',  'reduce to 50%, investigate this week'),
        (30, 'critical',  'suspend pending review'),
        (0,  'dead',      'pull offline, full root cause required'),
    ]

    def compute_health(
        self,
        ir_score: float,
        signal_score: float,
        param_score: float,
        regime_score: float,
    ) -> Tuple[float, str, str]:
        w = self.COMPONENT_WEIGHTS
        composite = (
            ir_score       * w['ir'] +
            signal_score   * w['signal'] +
            param_score    * w['params'] +
            regime_score   * w['regime']
        )
        score = round(composite, 1)

        for threshold, status, action in self.HEALTH_BANDS:
            if score >= threshold:
                return score, status, action
        return score, 'dead', 'pull offline, full root cause required'

what q1 looked like in hindsight
#

ran this retroactively against the SPX premium selling strategy for all of Q1. i’ve been watching that strategy almost entirely through the risk engine, not through a health lens. here’s what the health score shows:

Figure 1: Composite health score for SPX premium selling strategy across Q1 2026. Score dropped below 70 (degraded threshold) by Feb 9 — three weeks before my largest single-week drawdown. Regime alignment drove most of the degradation as sustained trending vol replaced the mean-reverting conditions the strategy was built for.

the uncomfortable thing: health dropped below 70 on Feb 9. that’s when the tariff headlines started running directional. i had risk engine monitoring at that point but wasn’t running health scores yet. if i had been, Feb 9 would have been the signal to cut allocation in half. instead i ran at full size through Feb and mid-March. the drawdown was survivable but it was bigger than it needed to be.

parameter drift under the hood
#

here’s the second thing this framework surfaced: my vol threshold parameter on the SPX premium selling strategy had drifted significantly by mid-feb. the parameter that was calibrated at 0.85 was now “wanting” to be around 1.3 based on the walk-forward optimization runs. that’s not small drift. that’s a 53% shift in a key parameter.

Figure 2: Parameter drift heatmap across Q1. Red = parameter “wants” to be higher than calibrated, blue = lower. Vol threshold (top row) shows the clearest trend — optimal value shifted +53% from calibration by mid-March. This confirms the strategy wasn’t broken, it was running miscalibrated settings for the high-vol trending conditions that defined Q1.

the lookback window and position sizing both show symmetric drift — the walk-forward runs consistently suggest a longer lookback and smaller positions, which makes sense for trending vol. the strategy was calibrated in a different regime.

this is actually not terrible news. drift doesn’t mean the strategy is dead. it means it needs retuning. that’s very different from signal decay where the edge itself has eroded.

what i’m actually doing with this for Q2
#

practical changes starting monday:

1. weekly health audit — every sunday night, run the health scorer against each strategy. composite score below 70? investigate before the week starts. below 50? reduce allocation or suspend until i understand why.

2. parameter check every two weeks — run the fast walk-forward. if any parameter drifts more than 20% from calibrated, flag for review. i’ll use optuna for this since it’s fast enough for weekend batch runs.

3. pre-Q2 recalibration — spending the next week retuning the SPX premium selling parameters using Q1 data as part of the training set. the health framework flagged the issue; now i need to fix it.

NexusFi has a solid thread on the mechanics of taking a systematic strategy from backtesting to live capital that gets into a lot of the edge-decay issues that come up once you’re running real money — Taking a Trading System Live is one of the threads i’ve gone back to multiple times since joining in 2023. also, the Academy has a deep-dive on strategy optimization and parameter tuning that goes into walk-forward methodology in a lot of detail.

Q2 starts now
#

chicago colo ran the overnight health score batch while i was writing this. results just hit the TimescaleDB table.

SPX premium selling: 52/100 (degraded, 50% allocation pending recalibration). BTC/ETH momentum: 74/100 (healthy, full allocation). ES mean reversion: 61/100 (degraded, investigating this week).

not ideal. but it’s honest. and honest is better than flying blind through another quarter like march.

a. is still asleep. i’m running strategy health checks on a monday at 3am. this is just the job.

Q2, let’s go.

-AK

Related

q1 close: final numbers, colo benchmarks, and q2 setup
friday night. Q1 officially in the books. did the math earlier while A. was cooking. she noticed i went quiet and just left me to it. that’s one of the things i didn’t expect about being married — how well she reads when to give space. anyway.
real-time greeks aggregation: knowing your portfolio delta/gamma at sub-second speed
2:15am wednesday. still processing this week. the q1 factor attribution post from sunday was cathartic but it also made me confront something i’d been papering over: i was flying blind on real-time greeks for most of march. not completely blind — i had position-level greeks from IB’s TWS feed. but aggregating them into a coherent portfolio view? that was a manual spreadsheet thing i’d run every few hours.
mean reversion implementation - statistical edge in practice
finally deploying the mean reversion algo I’ve been backtesting since june. 6 months of development. time to go live. the edge # simple concept: prices that deviate from their mean tend to revert.
q1 factor attribution: theta is the edge, delta drift is the problem
q1 is in the books. three months, roughly flat performance, and a clear pattern in the trade data that tells me exactly what needs to change for q2. jan: +2.1%. feb: -1.3%. march: -0.9% (locked at friday close). quarter: -0.13% net. account moved from $1.196M to about $1.194M. call it flat with a slight downside tilt.
march vol spike: when the risk engine earns its keep
2:30am friday. rough week in the books. march has been a whole thing. tariff headlines dropping every 48 hours, VIX spiking then partially recovering, nobody knows what SPX does next. january was decent (+2.1%), february went against me (-1.3%). march hasn’t been great either. week ending today, i’m down about $2.3k for the five sessions. month’s probably closing around -1%.
dynamic position sizing - kelly criterion meets regime detection
one of the dumbest things i did in 2023 was running fixed position sizes. every trade was the same size regardless of conviction, volatility, or recent performance. looking back it’s obvious why i hemorrhaged $180k - i was sizing up the same during high-vol crashes as during calm trending markets.