Core Architecture & Requirements:
- Scale: 10,000 writes/sec, 100,000 reads/sec (10:1 read-heavy ratio), sub-20ms p95 redirect latency.
1. Short Key Generation & Hash Strategy:
- Use Base62 encoding ([a-zA-Z0-9]) on a 64-bit integer ID. A 7-character string yields 62^7 (~3.5 trillion) unique URLs.
- Pre-generate IDs using a distributed ticket server or Twitter Snowflake cluster to guarantee unique monotonic keys without database collisions.
2. Storage Layer:
- Primary Datastore: NoSQL document or key-value store (e.g., DynamoDB or Cassandra) partitioned on short_key. Schema: { short_key (PK), original_url, user_id, created_at, expires_at }.
- Caching Layer: In-memory distributed cache (Redis) with LRU eviction storing the top 20% most accessed URLs, serving 80%+ of redirection traffic directly from memory.
3. Routing & Redirection Flow:
- Read: API Gateway -> Check Redis -> On cache miss, query Cassandra -> Populate Redis -> Return 302 Found (temporary redirect so analytics can be logged on each visit).
- Write: API Gateway -> Rate Limiter -> Snowflake ID generator -> Base62 encode -> Persist to DB -> Return short URL.
4. Analytics & Async Processing:
- Decouple click metrics using an append-only Kafka topic consumed by an asynchronous analytics aggregation service.