Distributed DB – Neuronix Lab

Architecting a Flash-Sale System for 100M+ Concurrent Users: Static/Dynamic Split, Atomic Redis Lua Deduction and Queue-Based Load Shedding

A systematic review of the layered defenses behind a flash-sale system that absorbs hundreds of millions of instantaneous requests: CDN static offload, gateway rate limiting, atomic stock pre-deduction with Redis + Lua, asynchronous persistence through RocketMQ, and inventory bucketing to escape hot-row locking in the database.

#High Concurrency #Rate Limiter #Redis Lua

stock_deduct.lualua

-- Redis Lua script: atomic stock pre-deduction with duplicate-order protection
local stockKey = KEYS[1]
local userOrderKey = KEYS[2]
local userId = ARGV[1]
local quantity = tonumber(ARGV[2])

-- 1. Has this user already ordered?
if redis.call('SISMEMBER', userOrderKey, userId) == 1 then
    return -1 -- Already ordered; block the duplicate
end

-- 2. Check remaining stock
local currentStock = ton

Michael Sheng 2025-02-16

Read

Distributed DB

Difficulty: Expert 14 min

PostgreSQL 17 and pgvector in Practice: Hundred-Million-Scale Vector Search and HNSW Index Tuning

As RAG architectures spread, the operational cost of running a dedicated vector database stays stubbornly high. This article covers using the pgvector extension in PostgreSQL 17, with HNSW (Hierarchical Navigable Small World) and IVFFlat indexes, to build an enterprise knowledge retrieval engine holding 10 million vectors on a single node with 10ms response times.

#HNSW #pgvector #PostgreSQL 17

pgvector_hybrid_search.sqlsql

-- PostgreSQL 17: create an HNSW vector index and run a fast hybrid search
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE tech_knowledge_base (
    id BIGSERIAL PRIMARY KEY,
    category VARCHAR(64),
    title TEXT,
    content TEXT,
    embedding vector(1536) -- OpenAI text-embedding-3 dimensionality
);

-- Build the HNSW index (m=16 neighbors, ef_construction=64 build depth)
CREATE INDEX O

Alexander Zhao 2025-02-04

Read

Distributed DB

Difficulty: Architect 16 min

Distributed Transactions Head to Head: Raft State Machine Replication vs. Spanner TrueTime

A deep dive into the core algorithms modern distributed relational databases such as Google Spanner, TiDB and CockroachDB use for multi-region, multi-replica transactions, comparing what Raft lease reads, Multi-Paxos and the TrueTime API each pay in clock-wait cost to guarantee external consistency (linearizability).

#ACID #Distributed Database #Go

hybrid_logical_clock.gogo

package consensus

import (
	"sync"
	"time"
)

// Hybrid logical clock: physical time combined with Lamport causal ordering
type HybridLogicalClock struct {
	mu sync.Mutex
	l  int64 // Physical time, high bits (milliseconds)
	c  int32 // Logical counter
}

func (h *HybridLogicalClock) Now() (physical int64, logical int32) {
	h.mu.Lock()
	defer h.mu.Unlock()

	pt := time.Now().UnixMilli()
	if pt >

Dr. Evelyn Vance 2025-01-15

Read