Inter-Process Communication: How Isolated Programs Actually Talk

The core problem: two address spaces, one message

A modern OS gives every process a private virtual address space backed by page tables. Process A's pointer 0x7fff... and process B's identical pointer resolve to different physical frames. This is enforced by the MMU in hardware, so A literally cannot dereference B's memory — the CPU raises a fault. That's the safety guarantee we want to keep.

IPC breaks the isolation on purpose, and there are only two fundamental strategies:

• Message passing — the data lives in the kernel. A copies bytes into a kernel buffer via a syscall; B copies them out via another syscall. Two copies, two crossings of the user/kernel boundary. The kernel is the trusted intermediary, so it can enforce ordering, buffering, and access control. Pipes, FIFOs, message queues, and sockets are all this.
• Shared memory — the kernel maps the same physical frames into both address spaces. After the one-time mmap/shmat setup, reads and writes are plain loads/stores with zero kernel involvement and zero copies. It's the fastest possible IPC — but the kernel is no longer mediating, so you must supply the synchronization (a mutex, semaphore, or lock-free protocol) or the two processes will race and tear the data.

The invariant that separates a correct channel from a corrupt one: a reader must never observe a partially written message. Message passing gets this for free (the kernel copies atomically up to PIPE_BUF bytes); shared memory makes it your problem.

Pipes and FIFOs: McIlroy's byte stream

The anonymous pipe, added to Unix by Doug McIlroy in 1973, is the archetype. pipe(fds) returns two file descriptors — fds[0] read end, fds[1] write end — connected to a kernel-resident circular buffer (default 64 KiB on Linux, tunable via fcntl(F_SETPIPE_SZ)). It is unidirectional and only usable between processes that share the descriptor, i.e. a parent and its forked children, because the child inherits the open FD table.

The shell wires a | b like this:

pipe(fd);                 // fd[0]=read, fd[1]=write
if (fork()==0) {          // child = producer 'a'
  dup2(fd[1], STDOUT);    // stdout -> pipe write end
  close(fd[0]); close(fd[1]);
  exec("a");
}
if (fork()==0) {          // child = consumer 'b'
  dup2(fd[0], STDIN);     // stdin  <- pipe read end
  close(fd[0]); close(fd[1]);
  exec("b");
}
close(fd[0]); close(fd[1]);  // parent holds neither end

The flow control is the buffer: a write blocks when the buffer is full; a read blocks when it is empty. When all write ends close, the reader gets EOF (read returns 0); when all read ends close, a writer gets SIGPIPE (the reason a broken pipe kills yes | head). Writes of ≤ PIPE_BUF (512 by POSIX, 4096 on Linux) bytes are atomic — they won't interleave with another writer's bytes — which is the guarantee that makes concurrent logging to one pipe safe.

A named pipe (FIFO), created by mkfifo, is the same object with a name in the filesystem, so unrelated processes can open it by path. Opening the read end blocks until a writer appears (and vice-versa) — a built-in rendezvous.

Message queues and sockets: typed and networked

Message queues preserve message boundaries (unlike a pipe's undifferentiated byte stream). POSIX mq_send/mq_receive keep messages sorted by priority in a queue; a receive returns exactly one whole message and always the highest-priority one. The older System V queues (msgsnd/msgrcv) let a receiver select by a long mtype field, so one queue can multiplex several logical channels. Both survive the death of the sender — they're kernel objects with their own lifetime, which is a footgun: leaked SysV queues persist until ipcrm or reboot.

Unix domain sockets (AF_UNIX) are the workhorse of local IPC. Same socket()/connect()/accept() API as TCP but no protocol stack, no checksums, no loopback routing — just a kernel buffer copy, so they're markedly faster than a TCP loopback connection. They add two things pipes can't do:

• Bidirectional, connection-oriented streams (SOCK_STREAM) or datagrams (SOCK_DGRAM) between totally unrelated processes, addressed by a filesystem path or an abstract name.
• Ancillary data via sendmsg/SCM_RIGHTS: you can pass an open file descriptor to another process. The kernel dups the FD into the receiver's table. This is how a listener process hands live connections to workers, and how Wayland/browsers pass GPU buffer handles. You can also pass verified peer credentials (SO_PEERCRED) — the kernel vouches for the peer's PID/UID, which sockets use for authentication.

This is why PostgreSQL, MySQL, Docker, Redis, and the X11/Wayland protocols all default to Unix sockets for local clients: one API, FD passing, and OS-enforced credentials.

Shared memory: zero copies, your synchronization

When throughput matters — video frames, database buffer pools, a market-data feed — you cannot afford two copies and two syscalls per message. Shared memory maps one set of physical frames into N processes. Setup on Linux/POSIX:

• Create a backing object: shm_open("/name", O_CREAT|O_RDWR) (a tmpfs file) or memfd_create, then ftruncate it to size.
• Map it: mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0) in each process. Now both hold pointers into the same frames.
• Communicate by plain load/store. No kernel, no copy, no syscall on the fast path.

The catch is that the kernel is out of the loop, so it can't serialize access. Two processes writing the same region race exactly like two threads. You must place your synchronization primitive inside the shared region: a pthread_mutex initialized with PTHREAD_PROCESS_SHARED, a POSIX named semaphore, or — the modern high-performance answer — a futex. The classic pattern is a shared-memory ring buffer (SPSC queue): a producer writes at head, a consumer reads at tail, each an index into a power-of-two array. With one producer and one consumer, a correct memory-barrier discipline makes it lock-free — no syscall at all in the steady state, only a futex wake when the queue transitions empty↔non-empty.

The two must-not-forget rules: publish the data before you publish the index (a release store), and read the index before you read the data (an acquire load) — otherwise the CPU or compiler reorders the store and the reader sees a stale slot. This is the same acquire/release fence discipline as multithreaded lock-free code; the address space just happens to be shared across processes.

The cost model: copies, crossings, and context switches

Every IPC's latency decomposes into a small set of countable costs. Getting this model right is the whole game in an interview and in production tuning.

• Data copies. Message passing = 2 (user→kernel on send, kernel→user on receive), each O(n) in message size n. Shared memory = 0. For a 1 MB message the copy dominates everything else, and the 2× copy is why bulk transfers move to shared memory.
• Mode switches (syscalls). A syscall is a trap into the kernel — hundreds of cycles, plus TLB/cache pollution. Message passing pays ≥ 2 per message; a well-designed shared-memory queue pays 0 on the fast path and only traps to futex_wait/futex_wake when it must actually block.
• Context switches. If the receiver was blocked, delivering a message wakes it, and the scheduler may switch — thousands of cycles plus cold caches. This is why batching (amortizing one wakeup over many messages) is the first optimization: it turns O(messages) switches into O(batches).

Rough Linux ballparks: a pipe/socket round-trip is ~1–5 μs, dominated by the two syscalls and the wakeup; a shared-memory handoff can be ~50–200 ns when no process blocks. The asymptotic cost of sending n bytes is O(n) for message passing versus O(1) amortized for shared memory (you copied nothing; you only updated an index). Space: a pipe is O(buffer); a shared segment is O(segment); a signal is O(1) — it carries no payload beyond a number.

Choosing a mechanism, and the ways it breaks

The decision reduces to a few questions:

• Stream or discrete messages? Byte stream → pipe or SOCK_STREAM socket. Typed/prioritized units → message queue or SOCK_DGRAM.
• Related or unrelated processes? Parent/child only → anonymous pipe. Unrelated → FIFO, named socket, or named shared memory.
• Throughput-bound? Bulk data (frames, buffer pools) → shared memory with your own ring buffer + futex.
• Need FD passing or peer auth? → Unix domain socket with SCM_RIGHTS/SO_PEERCRED. Nothing else offers it.
• Machines can be remote? → TCP/Unix sockets are the only options that generalize across the network.

The failure modes are the classic ones:

• Deadlock. A and B both write to a full pipe and neither reads → both block forever. The bidirectional-pipe deadlock is a canonical bug; the fix is nonblocking I/O with select/epoll, or draining before writing.
• SIGPIPE. Writing to a pipe/socket whose reader has closed raises SIGPIPE, which by default kills the process. Servers must signal(SIGPIPE, SIG_IGN) and handle EPIPE.
• Torn reads in shared memory. Forgetting the acquire/release fences or the mutex yields a reader that sees half-updated structs — nondeterministic, load-dependent corruption that vanishes under a debugger.
• Resource leaks. SysV queues/segments and POSIX shm_open objects outlive their creators; a crash leaks them until ipcrm/unlink. Always pair create with a cleanup path.
• Lost signals. Standard signals don't queue — two SIGCHLDs before you handle one collapse into a single delivery, so you must loop waitpid. Use signalfd or realtime signals if you need queuing.

IPC mechanisms: copies, latency, and synchronization built in

Mechanism	Kernel copies	Sync built in?	Best for
Pipe / FIFO	2 (write, read)	Yes (blocking)	Byte streams, related processes
Message queue	2	Yes (priorities, blocking)	Discrete typed messages
Unix domain socket	2	Yes	Streams + FD passing, unrelated procs
Shared memory	0	No — you add a mutex/futex	Bulk / high-throughput data
Signal	0 (just a number)	N/A (async)	Notifications, control events