Operating systems for backend

Processes, threads, and coroutines

A process has its own address space, file-descriptor table, and signal handlers. A thread shares all three with its siblings and keeps only its own stack. That shared address space is the tradeoff: threads talk through the same heap for free, but every shared write needs a lock, and one thread corrupting memory takes down the whole process. Coroutines (goroutines, async/await) switch in user space without a kernel trap, so a runtime can hold far more of them. A goroutine's stack starts around 2 KiB and grows on demand, while a default Linux thread reserves an 8 MB stack.

Wrong "Threads are always faster than processes." Your runtime often decides. Python's GIL runs one thread of bytecode at a time, so threads give I/O concurrency but no CPU parallelism; Node runs your code on a single event-loop thread, so one CPU-heavy handler stalls every request. Right pick on isolation versus sharing and on what the runtime parallelizes.

Why load average is not CPU percent

On Linux the load average counts running and runnable tasks plus tasks in uninterruptible sleep, which is mostly disk I/O. It measures system demand and carries no percent, so read it against the CPU count. A load of 12 on 8 vCPUs while the cores sit idle means threads are blocked on I/O, not that the CPUs are pegged. The three figures are 1/5/15-minute exponentially damped averages, so a series like 30/22/11 is a load still climbing. Confirm the cause with mpstat -P ALL 1 (%usr versus %iowait) before you act.

SIGTERM vs SIGKILL and graceful shutdown

docker stop and Kubernetes both send SIGTERM, wait a grace period (10s for Docker, 30s for Kubernetes by default), then send SIGKILL. SIGTERM is catchable, so your handler can drain in-flight requests and close connections; SIGKILL can never be caught, blocked, or ignored, so whatever it interrupts is lost. The common container bug is an sh -c "app" entrypoint: the shell runs as PID 1 and does not forward SIGTERM to your app, so the grace period expires and SIGKILL drops live requests. Run the app as PID 1 with the exec form, or use an init like tini (docker run --init).

Reading memory: available, not free

A large buff/cache in free -m is reclaimable page cache that the kernel hands back to programs on demand, so it is not memory that is used up. Read the available column. Linux also overcommits: malloc reserves virtual space that can exceed RAM, so it rarely returns NULL, and the shortfall surfaces later as an OOM kill when a page fault cannot be satisfied. In a container the limit that fires is usually the cgroup's memory.max, which OOM-kills inside that container without taking down the host.

What a context switch costs

A process switch reloads the page-table base register and can invalidate TLB entries, so the next memory accesses re-walk the page tables. A thread switch within one process keeps those page tables and skips that cost, though both threads and processes come from clone(2). A coroutine switch stays in user space and never traps into the kernel, avoiding the syscall and the TLB effects. That is why Go multiplexes many goroutines onto a small pool of OS threads (an M:N scheduler), getting cheap concurrency and real parallelism at once, while Python's GIL serializes bytecode and forces CPU parallelism into separate processes.

Overcommit and the OOM killer

Linux overcommits by default: allocations reserve virtual address space that may exceed RAM plus swap, on the bet that most of it is never touched. vm.overcommit_memory sets the policy: 0 (heuristic, the default), 1 (always allow), and 2 (strict, capping total commit at CommitLimit = swap + overcommit_ratio% of RAM, default ratio 50, and failing allocations past it with ENOMEM). Modes 0 and 1 defer the reckoning to fault time, where a page that cannot be provided invokes the OOM killer to kill a process by badness score. Bias survival with /proc/<pid>/oom_score_adj, range -1000 to +1000, where -1000 makes a process effectively unkillable; the legacy oom_adj interface (-17 to +15) is retired. Under cgroup v2, usage hitting memory.max invokes the OOM killer inside that cgroup, so one container dies without dragging down the host.

epoll: readiness, not completion

epoll is a readiness API. epoll_ctl maintains the interest list, and epoll_wait returns the ready list, the subset with pending I/O, so its cost scales with ready fds instead of scanning every watched fd the way select and poll do on each call. Level-triggered mode (the default) keeps reporting a fd while data remains and is the safe choice.

Wrong "With edge-triggered epoll I read once when notified." EPOLLET fires only on a state transition, so you must use non-blocking fds and read or write until EAGAIN; stop early and the leftover bytes never re-trigger, and the connection hangs. Completion APIs invert the model: io_uring on Linux and IOCP on Windows perform the I/O and hand back the finished result. Go's netpoller and Node's libuv both sit on epoll, kqueue, or IOCP, but libuv's thread pool (default 4) serves file I/O, DNS, and some crypto, not sockets, so heavy file or crypto work starves unrelated async calls.

Containers are namespaces plus cgroups, not VMs

A container is a host process with a restricted view: no guest kernel, no hypervisor. Namespaces govern what it can see (mount, PID, network, user, and more, each a CLONE_NEW* flag set via clone, unshare, or setns); cgroups v2 govern what it can use. Two memory knobs differ: memory.max is a hard limit that fires the in-cgroup OOM killer, while memory.high throttles and applies reclaim pressure but never invokes OOM. For CPU, cpu.max (format "MAX PERIOD" microseconds, default "max 100000") caps bandwidth, and its throttling shows as latency spikes even when average CPU looks low, so a tight limit can hurt more than a proportional cpu.weight share (default 100).