All articles
Performance6 min read2023-07-22

Distributed Redis Caching Strategy for High-Frequency API Calls

Our approach to caching real-time freight rate data with Redis — balancing freshness vs performance at 2000+ RPS.

Redis.NETCachingPerformanceDistributed Systems

The Problem

Real-time freight rates change, but not that fast. Carrier APIs were being hammered with the same origin+destination+container combination thousands of times per minute — mostly cache-miss scenarios serving identical data.

Caching Strategy

We implemented a layered cache:

Request → In-Memory L1 (10s TTL)
             ↓ miss
         Redis L2 (5min TTL)  
             ↓ miss
         Carrier API + write-through to Redis + L1

Implementation

public class CachedRateService : IRateService
{
    private readonly IDistributedCache _redis;
    private readonly IMemoryCache _local;
    private readonly IRateService _inner;
 
    public async Task<List<CarrierRate>> GetRatesAsync(RouteKey key)
    {
        var cacheKey = $"rates:{key}";
        
        // L1: in-process memory
        if (_local.TryGetValue(cacheKey, out List<CarrierRate>? cached))
            return cached!;
 
        // L2: distributed Redis
        var bytes = await _redis.GetAsync(cacheKey);
        if (bytes is not null)
        {
            var rates = MessagePackSerializer.Deserialize<List<CarrierRate>>(bytes);
            _local.Set(cacheKey, rates, TimeSpan.FromSeconds(10));
            return rates;
        }
 
        // Origin: fetch & populate both layers
        var fresh = await _inner.GetRatesAsync(key);
        var packed = MessagePackSerializer.Serialize(fresh);
        
        await _redis.SetAsync(cacheKey, packed, new DistributedCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5)
        });
        _local.Set(cacheKey, fresh, TimeSpan.FromSeconds(10));
 
        return fresh;
    }
}

Cache Invalidation

When a carrier pushed a rate update event, we used Redis pub/sub to invalidate across all nodes:

await _subscriber.PublishAsync(
    "rate-invalidation",
    JsonSerializer.Serialize(new { carrier = "MAERSK", lanes = affectedLanes })
);

Results

  • API cache hit rate: 94%
  • Carrier API calls reduced by ~90%
  • Average response time: 2,800ms → 85ms at P95
  • Redis cluster: 3-node, 6GB total — handling 2,000+ RPS comfortably