Hardware Transactional Memory: Optimistic Locking in Silicon

What It Is and the Problem It Solves

A lock is pessimistic: it assumes conflict is likely, so a thread grabs exclusive ownership before touching shared data, forcing everyone else to wait even when they would never have collided. That serialization is the enemy of many-core scaling, and fine-grained locking — a separate lock per hash bucket, per tree node — is notoriously hard to get right (deadlock, lost updates, priority inversion).

Hardware Transactional Memory takes the opposite bet, borrowing the idea of optimistic concurrency control (Kung & Robinson, 1981) and pushing it into the CPU. It was proposed by Maurice Herlihy and J. Eliot B. Moss in their 1993 paper "Transactional Memory: Architectural Support for Lock-Free Data Structures." The programmer marks a region as a transaction; the hardware runs it speculatively, buffering the writes so no other core can see them; and at the end it atomically commits (all writes appear at once) if nothing conflicted, or aborts (all writes vanish, registers roll back) if something did. The critical-section body looks exactly like the single-threaded code — the atomicity comes from silicon, not from careful lock ordering.

The Speculative Mechanism, Step by Step

Intel's implementation, TSX (Transactional Synchronization Extensions), exposes two interfaces. The explicit one, RTM (Restricted Transactional Memory), adds four instructions: XBEGIN, XEND, XABORT, and XTEST. A transaction proceeds like this:

• XBEGIN fallback — the CPU takes a checkpoint of the architectural register state and records a fallback address. Everything after this runs speculatively.
• Speculative reads add each touched cache line to the transaction's read set; the line is marked as read-monitored.
• Speculative writes add lines to the write set and are buffered in the L1 data cache in a special transactional state — not made globally visible, not written back to memory.
• XEND — commit. At a single instant the buffered write set becomes globally visible and the read/write marks clear. Because commit is one atomic step, other cores see either none of the transaction's writes or all of them.
• Abort — on any conflict or disallowed event, the CPU discards the buffered writes, restores registers to the XBEGIN checkpoint, and jumps to fallback with a status code in EAX explaining why.

Nested transactions are flattened: an inner XBEGIN is subsumed into the outermost one, and only the outermost XEND commits (nesting depth is bounded by an implementation limit). XABORT lets software force an abort with an 8-bit reason code, and XTEST reports whether execution is currently transactional.

Conflict Detection: Reusing the Cache-Coherence Protocol

The elegant part is that HTM needs almost no new machinery to detect conflicts — it piggybacks on the cache-coherence protocol (MESI/MESIF) that every multicore already runs to keep caches consistent. Each transactional line carries read/write monitoring bits. A conflict is exactly a coherence event that would invalidate a monitored line:

• Another core issues a write (read-for-ownership) to a line in your read set → your transaction aborts (someone changed data you depended on).
• Another core reads or writes a line in your write set → abort (they would see, or clobber, your uncommitted state).

Detection is therefore at cache-line granularity (64 bytes), which is efficient but causes false conflicts: two threads touching different variables that share a line (false sharing) will abort each other even though they never truly collided. Aborts also come from several non-conflict sources, and this is what makes RTM best-effort:

• Capacity — a monitored line gets evicted. The write set must fit in the L1 data cache (on Haswell, 32 KB, 8-way set-associative, 64-byte lines); nine write-set lines that map to the same set overflow the 8-way associativity and abort regardless of total size.
• Interrupts, context switches, page faults, and most system calls abort the transaction — architectural state cannot straddle a speculative boundary.
• Unsupported instructions (certain privileged, I/O, or serializing ops) and the explicit XABORT.

The EAX status after an abort encodes the reason in its low bits — e.g. bit 0 (explicit XABORT), bit 1 (retry may succeed), bit 2 (conflict), bit 3 (capacity) — so the fallback code can decide whether to retry or give up.

Lock Elision and the Mandatory Fallback

The killer application is lock elision: run a lock-protected critical section transactionally and only truly acquire the lock if a conflict forces you to. TSX's second interface, HLE (Hardware Lock Elision), makes this backward-compatible via two instruction prefixes. XACQUIRE on the lock-acquire instruction tells the CPU to elide the write that would take the lock — instead it adds the lock word to the read set and enters the critical section speculatively — and XRELEASE commits. Because the lock is never actually written, many threads execute the same critical section concurrently as long as their data sets are disjoint. Only a real data conflict aborts them, at which point they retake the lock for real. Old binaries even run on non-TSX CPUs because the prefixes decode as no-ops.

Two rules are non-negotiable. First, there must always be a non-transactional fallback path, because RTM guarantees nothing: a transaction that keeps hitting capacity limits or a page fault could abort forever, so software wraps XBEGIN in a bounded retry loop and, after N failures, takes an ordinary lock. Second is lock subscription: a speculative execution must read the lock variable inside the transaction and abort if it is held. Otherwise a thread on the fallback path could hold the real lock while a speculative thread simultaneously commits changes to the same data — breaking mutual exclusion. Reading the lock adds it to the read set, so any real acquisition of it by another thread aborts every speculator. A downside is the lemming effect: once one thread falls back to the lock, it aborts all the speculators, who then also fall back, collapsing to serial execution until the storm clears.

Complexity, Costs, and When It Wins

Commit is conceptually O(1) — a single atomic transition — but the fixed overhead of XBEGIN/XEND is on the order of tens of cycles, so wrapping a trivial one-word update in a transaction is slower than a plain compare-and-swap. HTM pays off when a critical section touches several locations that a single CAS cannot cover atomically, yet remains small enough to fit the hardware limits.

Those limits define the whole cost model. The write set is bounded by L1D capacity and associativity (roughly a few hundred lines at best, far fewer if they collide in one cache set); the read set can extend somewhat further but is still finite. A transaction whose footprint exceeds this simply cannot commit and always falls back. Every abort is pure waste — the speculative work is thrown away and re-run — so HTM's expected cost is (successful work) + (abort rate × wasted work + fallback cost). The regime where it wins is therefore sharply defined: read-mostly or low-contention workloads with small critical sections and rare true conflicts — concurrent hash tables, trees, and skip lists where operations usually touch disjoint keys. Under high contention or with large footprints, the abort rate dominates and a good lock beats it.

Real Systems, History, and the Security Reckoning

The lineage runs from Herlihy & Moss's 1993 architecture proposal, through Software Transactional Memory (Shavit & Touitou, 1995) which emulated it with logs and read/write barriers at a heavy cost, to real silicon. Sun's Rock processor had HTM but was cancelled in 2009. IBM shipped it in Blue Gene/Q (2012) and POWER8 (2014, with suspend/resume and rollback-only transactions). Intel brought it mainstream with TSX on Haswell (2013). On the software side, glibc added transactional lock elision for POSIX mutexes on TSX hardware, and researchers used TSX to accelerate in-memory databases and concurrent data structures.

HTM's history is also a cautionary tale about speculation. Intel had to disable HLE/TSX via microcode on Haswell/early Broadwell in 2014 after a functional erratum. Far more seriously, in 2019 researchers disclosed TSX Asynchronous Abort (TAA, CVE-2019-11135) — a Meltdown/MDS-class transient-execution side channel in which the speculative work inside a transaction that later aborts still leaves microarchitectural traces (in load buffers) that a co-resident attacker can sniff. The fix was again microcode, and Intel ultimately deprecated and disabled TSX by default across much of its product line from 2021 onward, with later cores dropping it entirely. It is a vivid reminder that the same speculation that makes optimistic execution fast also makes it a leakage surface — the family of problems that also underlies Spectre and Meltdown.

How It Differs From the Look-alikes

HTM is often confused with its neighbors. Compare-and-swap is atomic over exactly one word (or a double-word); HTM generalizes that to an atomic multi-location read-modify-write bounded only by cache capacity — you can think of it as "multi-word CAS with automatic conflict detection." Lock-free data structures built from CAS guarantee system-wide progress by construction; HTM guarantees nothing on its own, which is why the fallback lock is mandatory and why HTM is called best-effort. Software TM offers the same programming model with unbounded transactions but pays a large constant factor for instrumented accesses and version logs; HTM is cheap but bounded, so hybrids ("hardware-accelerated STM") try to use HTM for the common case and STM when transactions overflow.

Finally, HTM's optimistic-then-validate structure is the shared-memory sibling of database concurrency: optimistic concurrency control, snapshot isolation, and multiversion schemes make the same bet at the storage layer, validating a transaction's read set at commit and aborting on conflict. HTM simply implements that idea in nanoseconds, using cache coherence as its validation oracle rather than a version-number check.

HTM versus other ways to make a multi-word critical section atomic

Mechanism	Atomicity scope	Concurrency under low contention	Progress guarantee	Overhead / cost
Mutex / spinlock	Whole critical section	None (serialized)	Blocking, but always proceeds	Low base cost; serializes everything
Compare-and-swap (CAS)	One word (or double-word)	High for single-location updates	Lock-free (some thread progresses)	Cheap, but only one location
Software TM (STM)	Arbitrary read/write set	High, composable	Depends on contention manager	Heavy — instrumented reads/writes, logs
Hardware TM (HTM)	Read/write set up to cache capacity	High — many threads run in parallel	Best-effort (no guarantee a txn commits)	~Tens of cycles per begin/commit; abort wastes work