can’t fix what you don’t measure. built a grafana dashboard to track everything my algos are doing.
why i need this #
when strategies shit the bed, i need to know immediately. not 2 hours later when i check my phone and see -$5k.
also need historical data to analyze what went wrong. “my algo lost money” isn’t useful. “my algo’s sharpe dropped from 1.2 to 0.3 on april 12 at 10:47am” is actionable.
the stack #
prometheus - scrapes metrics every 5 seconds grafana - visualizes everything in real-time dashboards postgres - stores trade history for longer-term analysis
running all this on my home server (dell r730 in the closet). total cost: $0/month after initial $3k hardware investment.
what i’m tracking #
broke it into 4 dashboard categories:
1. account metrics
- total equity (live updating)
- daily P&L
- buying power used
- margin utilization %
2. strategy performance
- P&L by strategy
- trades executed per strategy
- win rate (rolling 30 trades)
- sharpe ratio (rolling 30 days)
- current drawdown vs max drawdown
3. execution quality
- order fill time (target <100ms)
- slippage per trade
- rejected orders
- API latency to IB
4. system health
- CPU/RAM usage
- network latency
- database query time
- python process status
custom metrics in python #
instrumented all my strategy code with prometheus client:
from prometheus_client import Counter, Histogram, Gauge
import time
# Define metrics
trades_executed = Counter('trades_executed_total', 'Total trades executed', ['strategy', 'direction'])
trade_pnl = Histogram('trade_pnl_dollars', 'P&L per trade in dollars', ['strategy'])
position_size = Gauge('position_size_current', 'Current position size', ['strategy', 'symbol'])
slippage = Histogram('slippage_ticks', 'Slippage in ticks', ['order_type'])
class Strategy:
def execute_trade(self, symbol, direction, quantity):
start_time = time.time()
# Place order
order = self.broker.place_order(symbol, direction, quantity)
# Track execution time
execution_time = time.time() - start_time
# Record metrics
trades_executed.labels(
strategy=self.name,
direction=direction
).inc()
# Calculate slippage
expected_price = self.get_mid_price(symbol)
actual_price = order.filled_price
slippage_ticks = abs(expected_price - actual_price) / 0.01
slippage.labels(order_type='market').observe(slippage_ticks)
# Update position gauge
position_size.labels(
strategy=self.name,
symbol=symbol
).set(order.filled_quantity)
return order
alerts that saved my ass #
set up alertmanager to send telegram messages when:
- account down >2% in single day
- any strategy sharpe drops below 0.5
- slippage >10 ticks on any trade
- system latency >500ms
- python process crashes
already caught 2 bugs this week from alerts:
- strategy tried to trade after market close (rejected orders spiked)
- database connection leaked, queries taking 5+ seconds
seeing patterns in real-time #
most useful: correlating performance drops with specific events
example: noticed sharpe ratio tanking every tuesday 10am. pulled up order logs. realized that’s when economic data releases, spreads widen, slippage spikes.
solution: pause trading 9:55-10:05am on econ release days. saved probably $500/week in slippage costs.
cost: $0 ongoing #
prometheus + grafana are free. postgres is free. running on hardware i already own.
only cost is time to set it up (~8 hours) and maintain dashboards (~1 hour/week).
compare to commercial solutions like trading view pro ($60/month) or proprietary platforms ($200-500/month). fuck that.
sharing grafana config #
might open source my grafana dashboard configs on github if anyone’s interested. pretty specific to my strategies but could be useful template.
3:05am. dashboards are looking sick. now i can see my losses in beautiful real-time graphs instead of ugly broker statements.
-AK