Sockets: The Kernel Endpoint Behind Every Network Connection

The core idea: a socket is a named endpoint plus two queues

A socket is a kernel data structure representing one end of a communication channel. You create it with socket(domain, type, protocol) — e.g. socket(AF_INET, SOCK_STREAM, 0) for IPv4 TCP — and the kernel hands back a small non-negative integer, the file descriptor, an index into your process's file table. From that moment the socket looks like a file: you read(), write(), and close() it.

The load-bearing invariant is the connection identity. A connected TCP socket is uniquely keyed by the 5-tuple (protocol, local IP, local port, remote IP, remote port). Two different browser tabs to the same server share your IP and the server's IP:port but get different local ephemeral ports, so their tuples differ and the kernel demultiplexes arriving packets to the correct socket. This is why a server on port 443 can hold thousands of simultaneous connections on one port — the tuples are all distinct.

• Send buffer (write queue): bytes you've written but the peer hasn't yet acknowledged; drained by the TCP stack as ACKs arrive.
• Receive buffer (read queue): in-order bytes the stack has accepted but your app hasn't yet read(); its free space is advertised as the TCP receive window, giving flow control for free.

TCP is a byte stream, not a message stream: one write() of 4 KB may arrive as three read()s, and vice versa. Framing (length prefixes, delimiters) is your job. UDP sockets (SOCK_DGRAM) instead preserve datagram boundaries but drop the connection abstraction — no buffers-as-window, no retransmission.

How a connection is built, step by step

The server and client walk different sequences of syscalls. On the server:

• bind(fd, addr) — pin the socket to a local IP:port (e.g. 0.0.0.0:8080).
• listen(fd, backlog) — mark it passive and allocate two kernel queues: the SYN queue (half-open handshakes) and the accept queue (completed handshakes waiting to be handed to the app). backlog bounds the accept queue.
• accept(fd) — pop one completed connection off the accept queue and return a brand-new descriptor for it. The listening socket keeps listening; the new socket carries the connected 5-tuple. This split is the whole trick behind concurrent servers.

On the client: connect(fd, serverAddr) triggers the TCP three-way handshake (SYN → SYN-ACK → ACK). For a blocking socket connect() returns when the handshake completes; for a non-blocking one it returns EINPROGRESS and you wait for writability. Minimal server skeleton:

fd = socket(AF_INET, SOCK_STREAM, 0);
setsockopt(fd, SO_REUSEADDR, 1);
bind(fd, &addr);
listen(fd, 512);
while (1) {
  conn = accept(fd, &peer);   // new fd per client
  handle(conn);               // read()/write() the 5-tuple
}

Under the hood each connected socket rides the TCP state machine: CLOSED → LISTEN, then per connection SYN_RCVD → ESTABLISHED, and on teardown FIN_WAIT_1 → FIN_WAIT_2 → TIME_WAIT (the active closer lingers ~2·MSL, typically 60 s, so late duplicate segments can't corrupt a fresh connection reusing the tuple). TIME_WAIT accumulation is a classic ops headache on busy clients.

Complexity: why the readiness model, not the byte count, dominates

The interesting cost is not moving bytes — a read()/write() is O(k) in the bytes copied — it is discovering which of n sockets are ready so one thread can serve many. Three regimes:

• Thread-per-connection: conceptually trivial, but each thread costs a kernel stack (default ~1–8 MB of address space) and a scheduler slot. At n = 100,000 that's tens of GB of stacks and heavy context-switch churn. Time to serve is fine; space is O(n) with a huge constant, and this is exactly the wall the C10K problem named.
• select()/poll(): one thread, but every call passes all n descriptors into the kernel and the kernel scans all n. Cost is Θ(n) per wait, so serving n connections that each do a little work is Θ(n²) overall. select() also caps at FD_SETSIZE = 1024.
• epoll/kqueue: you register interest once — epoll_ctl(ADD) is O(1) (a red-black tree insert, O(log n), but effectively constant) — and epoll_wait() returns only the k ready descriptors in O(k), independent of total n. The kernel maintains a ready-list; readiness is delivered, not polled.

So epoll turns the per-tick cost from Θ(n) to Θ(k), where k = ready sockets. When only a handful of a million idle connections have data — the common case for chat servers, MQTT brokers, push gateways — that's the difference between a spinning CPU and a bored one. The memory footprint drops to O(n) small kernel structs plus one thread, not O(n) thread stacks.

Blocking, non-blocking, and the event loop

By default a socket is blocking: read() sleeps the thread until bytes arrive, write() sleeps until the send buffer has room. Simple, but one slow client stalls the thread. Set O_NONBLOCK (via fcntl) and those calls instead return immediately with EAGAIN/EWOULDBLOCK when they'd block. Non-blocking sockets are the substrate of every high-performance server.

The canonical pattern is the event loop: a single thread parks in epoll_wait(), wakes with a batch of ready fds, and services each without ever blocking. This is Nginx, Redis, Node.js (libuv), HAProxy, Envoy. Two epoll modes matter:

• Level-triggered (default): epoll_wait reports a socket as long as data remains. Forgiving — you can read a little and come back.
• Edge-triggered (EPOLLET): reported only on a transition (empty → data). You must drain the socket in a loop until EAGAIN, or you'll miss data. Fewer wakeups, higher throughput, less forgiving — a top interview gotcha.

ep = epoll_create1(0);
epoll_ctl(ep, EPOLL_CTL_ADD, listen_fd, {EPOLLIN});
while (1) {
  n = epoll_wait(ep, evs, MAX, -1);   // O(ready)
  for (i=0; i<n; i++)
    if (evs[i].fd == listen_fd) accept_all();
    else drain_until_EAGAIN(evs[i].fd);
}

The modern alternative is io_uring (Linux 5.1+): instead of asking "which fds are ready" you submit operations ("read 4 KB into this buffer") to a shared ring and reap completions, amortizing away the syscall per I/O entirely. It's a completion model, not a readiness model — closer to Windows IOCP.

Real systems and the tricks they use

Every networked program you touch is sockets underneath: web servers, databases, SSH, DNS, game netcode, gRPC. The design differences are almost entirely about how they multiplex:

• Nginx / Envoy / HAProxy: a small pool of worker processes, each an epoll (or kqueue) event loop, typically one per CPU core. No thread-per-connection.
• Redis: famously (near-)single-threaded event loop — its speed comes from never context-switching for network work.
• Node.js: libuv wraps epoll/kqueue/IOCP into one cross-platform loop; your JS callbacks run when a socket is ready.
• Go: hides all of this — a goroutine per connection does a blocking read(), but the runtime's netpoller parks it on epoll and multiplexes millions of goroutines onto a few OS threads. Best of both models.

Two crucial scaling options: SO_REUSEPORT lets multiple processes each bind() the same port; the kernel load-balances incoming connections across them by hashing the 4-tuple, eliminating the accept-lock contention (the "thundering herd") that plagued shared-listener designs. SO_REUSEADDR lets a restarting server rebind a port stuck in TIME_WAIT. And Nagle's algorithm (coalescing small writes) is why interactive protocols set TCP_NODELAY — the classic Nagle-vs-delayed-ACK 40 ms latency stall.

Pitfalls, edge cases, and the failure modes that bite

Sockets are simple until they aren't. The recurring bugs:

• Short reads/writes: write(fd, buf, 8000) may return 3000. You must loop on the returned count. Assuming a single write() ships everything is the #1 beginner bug, and it hides until buffers fill under load.
• Partial framing: because TCP is a stream, you need length-prefix or delimiter framing; a recv() can straddle two logical messages or split one.
• SIGPIPE: writing to a socket the peer already closed raises SIGPIPE, killing your process by default. Ignore it or use MSG_NOSIGNAL.
• Half-open connections: if a peer's machine yanks its power cord, TCP has no keepalive by default — your socket sits in ESTABLISHED forever. Enable SO_KEEPALIVE or app-level heartbeats.
• Ephemeral port / TIME_WAIT exhaustion: a client opening many short-lived connections can burn through the ~28,000 ephemeral ports, all stuck in TIME_WAIT. Reuse connections (pools, HTTP keep-alive) or tune the range.
• fd leaks: forgetting close() exhausts the per-process RLIMIT_NOFILE; accept() then fails with EMFILE and the server silently stops accepting.
• Edge-triggered under-drain: with EPOLLET, not looping to EAGAIN leaves bytes unread with no future wakeup — a hang that only shows under specific timing.

Edge case worth naming: accept() can itself return errors like ECONNABORTED (client hung up mid-handshake) — a robust loop treats it as "skip and continue," never fatal.

I/O readiness models: how one thread learns which of n sockets are ready

Mechanism	Per-call cost	FD limit	Trigger / Notes
select()	O(n) scan of bitmask	FD_SETSIZE = 1024	Level; rebuilds fd_set every call
poll()	O(n) scan of array	None (array grows)	Level; still passes all fds each call
epoll (Linux)	O(ready) return, O(1) add/mod	~open-files limit	Level or edge (EPOLLET); kernel keeps the set
kqueue (BSD)	O(ready) return, O(1) register	~open-files limit	Edge/level; filters beyond fds too
io_uring	O(batch), amortized syscall-free	~open-files limit	Completion (async op), not readiness