Concurrency – Neuronix Lab

A close look at why the classic epoll reactor drowns in syscalls and memory copies at a million QPS, followed by a hands-on walkthrough of building a lock-free, zero-copy network engine in Rust on Linux io_uring that sustains 4.5M requests per second on a single core.

#Concurrency #io_uring #Linux Kernel

iouring_zero_copy_engine.rsrust

use io_uring::{opcode, types, IoUring};
use std::os::unix::io::RawFd;

pub struct ZeroCopyServer {
    ring: IoUring,
    socket_fd: RawFd,
}

impl ZeroCopyServer {
    pub fn new(fd: RawFd, queue_depth: u32) -> std::io::Result<Self> {
        let ring = IoUring::builder()
            .setup_sqpoll(2000) // Enable the kernel SQPOLL thread with a 2ms idle timeout
            .setup_coop_taskrun()

Lucas Richter 2025-02-10

Read

AI & LLM Systems

Difficulty: Advanced 13 min

Python 3.13 Without the GIL: Free-Threading and FastAPI for Ten-Thousand-Connection Microservices

Python 3.13 ships an experimental free-threaded build that removes the Global Interpreter Lock (GIL) entirely. We benchmark multi-threaded CPU-bound work alongside FastAPI async I/O on a 32-core server and measure a 28x linear throughput gain.

#Concurrency #FastAPI #Free-Threading

fastapi_nogil_service.pypython

import asyncio
from concurrent.futures import ThreadPoolExecutor
from fastapi import FastAPI
import uvicorn

app = FastAPI(title="Neuronix No-GIL High-Throughput Engine")
executor = ThreadPoolExecutor(max_workers=32)

def heavy_cpu_matrix_math(n: int) -> float:
    # On Python 3.13 (no-GIL), 32 threads really can saturate 32 CPU cores at once!
    total = sum(i * i for i in range(n))
    return fl

Dr. Evelyn Vance 2025-01-20

Read

Millions of Concurrent Connections: A Zero-Copy Async Network Engine in Rust and io_uring

Python 3.13 Without the GIL: Free-Threading and FastAPI for Ten-Thousand-Connection Microservices