Skip to main content

upgrading chicago colocation to 10gbe - latency improvements

chicago colocation server needed upgrade.

1gbe connection = bottleneck.

current setup
#

location: chicago datacenter (equinix CH1)

server: dell poweredge r640

  • dual xeon gold 6248r (48 cores total)
  • 256gb ram
  • 4x 1tb nvme ssd (raid 10)

connection: 1gbe dedicated port

latency to exchanges:

  • CME (chicago): 0.8ms
  • CBOE (chicago): 1.2ms
  • NYSE (new jersey): 4.5ms

cost: $450/month (server + power + 1gbe port)

the problem
#

momentum strategy needs fast execution.

breakouts happen fast. milliseconds matter.

1gbe maxes out at ~125 MB/s.

when multiple positions exit simultaneously = packet queue.

estimated losing 50-150ms during high volume periods.

10gbe upgrade details
#

new connection: 10gbe dedicated port

bandwidth: 1.25 GB/s (10x faster)

latency improvement: removes packet queuing bottleneck

cost increase: +$200/month ($450 → $650/month)

cost-benefit analysis
#

cost: $200/month = $2,400/year

benefit calculation:

assume 50ms average latency reduction.

momentum strategy: 20 trades/month.

10 trades = entry, 10 trades = exit.

50ms * 20 trades = 1,000ms saved monthly.

slippage reduction:

assume 0.01% slippage reduction per trade.

20 trades/month * $2,000 avg position * 0.01% = $4/month saved.

wait that math sucks.

better justification
#

real benefit: reliability not speed.

1gbe packet loss during volatility = missed fills.

10gbe = headroom for future growth.

planning to add more strategies.

more strategies = more simultaneous orders.

10gbe handles 50+ concurrent connections easily.

also: professional setup looks better.

if ever pitch prop firms or investors, infrastructure matters.

$650/month colocation = serious trader.

$450/month = maybe serious.

migration plan
#

step 1: provision new 10gbe port (datacenter does this)

step 2: install 10gbe nic in server (sfp+ dual port)

step 3: configure network settings

step 4: test latency and throughput

step 5: cutover during weekend (low volume)

step 6: monitor for issues

datacenter confirmed 3-day turnaround.

scheduled for sept 20-23 (next week).

network configuration
#

# install 10gbe nic drivers
modprobe ixgbe

# configure network interface
ip link set ens4 down
ethtool -s ens4 speed 10000 duplex full autoneg off
ip addr add 10.50.12.148/24 dev ens4
ip link set ens4 up

# verify connection
ethtool ens4 | grep Speed
# should show: Speed: 10000Mb/s

# test bandwidth
iperf3 -c test-server.equinix.com -t 60 -P 10
# should show ~9.4 Gbps (close to 10gbe theoretical max)

expected latency improvements
#

current latency chain:

  1. algorithm detects signal: ~5ms
  2. order generation: ~2ms
  3. network transmission (1gbe): ~15ms
  4. broker processing: ~8ms
  5. exchange matching: ~3ms

total: ~33ms

after 10gbe upgrade:

  1. algorithm detects signal: ~5ms
  2. order generation: ~2ms
  3. network transmission (10gbe): ~3ms (12ms improvement)
  4. broker processing: ~8ms
  5. exchange matching: ~3ms

total: ~21ms (36% faster)

testing methodology
#

will run A/B test first week.

test 1: paper trade 50 signals on 1gbe

test 2: paper trade same 50 signals on 10gbe

metrics:

  • fill latency (order sent → fill confirmed)
  • slippage (expected price vs actual fill)
  • packet loss during volatility spikes

if improvement >10%, worth the cost.

if improvement <5%, maybe overkill.

home network for comparison
#

san diego setup:

dell poweredge r740xd at home.

ubiquiti 10gbe switch.

att fiber 5gbe internet.

home → chicago latency: ~45ms.

why colocation still matters:

45ms home-to-chicago.

0.8ms chicago-to-CME.

45ms vs 0.8ms = 56x faster.

colocation for execution.

home for research and backtesting.

infrastructure costs summary
#

chicago colocation (after upgrade):

  • server + power + 10gbe: $650/month

san diego home:

  • att fiber 5gbe: $180/month
  • power for servers: ~$120/month

data feeds:

  • polygon.io professional: $199/month
  • thetadata: $50/month

total infrastructure: $1,199/month = $14,388/year

trading capital: $360k

infrastructure as % of capital: 4% annually

reasonable for professional setup.

comparison to alternatives
#

cloud vps (aws/gcp):

  • cost: $800-1200/month for equivalent specs
  • latency: worse (shared infrastructure)
  • control: limited (their rules)

no colocation:

  • cost: $0/month
  • latency: 45ms+ from home
  • missed opportunities: high

colocation worth it for serious algo trading.

ROI calculation
#

if 10gbe upgrade improves fills by 0.01% on 50% of trades:

20 trades/month * $2,000 avg * 50% affected * 0.01% = $2/month.

wait that’s $24/year vs $2,400/year cost.

fuck. math doesn’t support upgrade.

real reason i’m doing it
#

headroom.

planning to scale up.

more strategies = more orders = more bandwidth needed.

1gbe is fine today.

10gbe is insurance for tomorrow.

also i like having best infrastructure.

can afford it. trading profitable. worth it for peace of mind.

A’s reaction
#

showed her the upgrade plan yesterday.

A: “you’re spending $2,400/year to maybe save $24?”

me: “it’s about headroom. future-proofing.”

A: “or you just like having a 10gig pipe.”

me: “…that too.”

she gets it. she’s a developer. understands over-engineering for fun.

next steps
#

sept 20: datacenter provisions 10gbe port

sept 21: install 10gbe nic, configure network

sept 22: test and validate latency improvements

sept 23: cutover production trading to 10gbe

sept 24-30: monitor performance, collect metrics

will post results in october.


4:31am saturday. upgrading chicago colocation to 10gbe next week. overkill but worth it for headroom. $650/month total cost.

-AK

Related

migrating market data to timescaledb - 10x query speedup
been storing market data in regular postgres. works but slow for time-series queries. migrated to timescaledb this week. 10x speedup on historical queries. got the idea from NexusFi algo infrastructure discussions about optimizing market data storage. someone mentioned timescaledb and i researched it.
refactored data pipeline to async - 3x faster market data processing
been running synchronous data fetching since january. works but slow during market open. refactored to async this week. 3x speed improvement. the problem with sync code # # Old synchronous approach def fetch_market_data(symbols): results = [] for symbol in symbols: data = fetch_from_api(symbol) # Blocks here results.append(data) return results # With 10 symbols, takes 10 * 180ms = 1,800ms total each API call blocks until complete.
added redis caching - cut market data latency by 60%
been noticing market data latency creeping up. average fetch time: 180ms from polygon API. slowing down entry execution. the problem # every time algo needs current price:
A. came over - gave her the server rack tour
A. came over sunday afternoon. gave her full tour of my trading setup. she fucking loved it. the setup tour # server rack:
rebuilt backtesting pipeline - 10x faster parameter optimization
spent last 3 days rebuilding backtest optimization pipeline. went from 6 hours to 35 minutes for full parameter sweep. the problem # old approach: sequential parameter testing.
using python async for real-time market data
rewrote my market data pipeline to use async. 3x faster, way cleaner code. the problem # old synchronous code: