🤠

In 2015 I formed a small group of engineers at Jane Street to rebuild the firm’s core trading system from the ground up, and we ended up cutting latency by two orders of magnitude. Some of the techniques we used, relevant for algorithmic trading systems and exchanges today: Zero allocation: Whenever a program allocates memory for an object on the heap, the runtime pays a steep penalty in latency. The simplest solution is to avoid memory allocation entirely. Jane Street famously uses OCaml, a strongly typed programming language that by default produces garbage collected by a dynamic collector. Most other firms use languages with manual memory management, but it was a strict part of Jane Street’s tech culture that all risk-sensitive code had to be written in OCaml. It took a collaborative effort across multiple groups within Jane Street’s technology org to create zero-allocation core libraries, combining the type safety of a functional programming language with the memory profile of a language like C. We built the new main trading loop in this hybrid OCaml/C-style, producing zero new allocations in the critical path from tick to trade. In modern languages like Rust, it is substantially easier to achieve precise memory management while still benefiting from type safety and compile-time guarantees. Kernel bypass: A primary goal of a low-latency trading system or exchange is to pull a network packet containing market data or order flow through the network card’s interface and into the program’s memory space as fast as possible. The standard Linux OS kernel uses slow abstractions to support a wide variety of network drivers, at the expense of the entire system’s end-to-end latency. When we started with an empty program that contained no business logic and only forwarded packets through when received, the end-to-end latency was already too slow. To fix this issue, we employed a standard practice in the HFT industry in which we bypassed the OS’s kernel stack entirely by leveraging our network card vendors’ proprietary APIs to DMA packets straight from the NIC into memory. This technique brought our empty-packet-forwarding baseline into the latency regime we needed in order to build out the rest of the trading, risk, and protocol code. Local IPC: Kernel bypass is necessary when reading routed packets off a network from a third party such as another exchange or client connection. When communicating between internal instead of external processes, the fastest transports avoid network stacks entirely. Processes within the same box can transfer messages using shared memory or Unix domain sockets. This allowed us to continue with our familiar process boundaries for separable components without sacrificing significant performance. We had to write custom logic to emulate many of the features of network- and transport-layer protocols, with the result of creating a reusable, zero-overhead IPC mechanism. Working on this problem was one of the most intellectually rewarding experiences of my early career. The above latency optimization techniques are fairly commonplace in the HFT trade but hard to learn outside the industry setting. Half of our team at Architect comes from Jane Street and other trading firms, and we value using our domain knowledge to build exchanges for the public rather than trading software that never leaves an HFT’s walls.