Most backends talk to themselves like they're calling a stranger on the internet. One user request turns into 5-10 hops inside the cluster, and every hop is JSON over HTTP. But you control and ship both sides. Nobody in there needs pretty field names, and you parse the same text again and again. The question is just where the call comes from. Outside the cluster: REST. Between your own services: gRPC. Read it this week on @EngPolymathic

Your latency benchmark shows a great p99. It might still be lying to you. The most popular way to measure latency is to wait for each response before sending the next request, which inadvertently coordinates with the system being measured. Think about a garbage collection pause. The system freezes for 100 ms, and a real workload would queue up dozens of slow requests. But the benchmark just logs a single one and moves on. Your p99 looks great, but it’s a lie. This is known as coordinated omission. I wrote the second article in my latency series that also covers where all that variance comes from, from CPU caches to garbage collectors, how latency compounds across services, and a small ping tool you can run to see your own network’s tail. Give it a read in this week’s issue of @EngPolymathic

What’s the difference between a blocking and a 𝐧𝐨𝐧-𝐛𝐥𝐨𝐜𝐤𝐢𝐧𝐠 algorithm? The best way to understand it is to start with blocking algorithms and continue with non-blocking ones. A blocking algorithm does one out of two things: - performs the action requested by a thread - blocks the thread until the action can be done safely If a thread (B) acquires the lock first and gets suspended, another thread (A) must wait for an arbitrary time. A non-blocking algorithm does one out of two things: - performs the action requested by the thread - notifies the requesting thread that the action can’t be done If a thread (B) acquires the lock first and gets suspended, another thread (A) gets turned down. The 𝐝𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐜𝐞 between blocking and non-blocking algorithms is in the second step. A blocking algorithm stops A until the lock is released, while a non-blocking algorithm notifies A that access is rejected. The most common way to implement a non-blocking algorithm is by implementing the data structure using CAS operations. CAS stands for “Compare and Swap”, and is an atomic operation. It compares the value of a variable to an expected value. If they do, it gives the variable a new value. Specific CPU instructions typically support CAS operations. Therefore, no synchronization or thread suspension is necessary. Using a non-blocking algorithm in a concurrent environment brings two main benefits: - No thread suspension. Thread B has a lower latency. It receives a response immediately and can decide what to do next. - No deadlocks. Thread B does not wait for the lock to release, and there is no possibility that a deadlock occurs.

Hash maps and hash sets show up in interviews all the time. The good news is that there are only two scenarios for using them. 1. Existence Checking whether an element exists in an array is a linear-time operation. Hash sets make it possible to do the check with amortized constant time. Suppose you have an algorithm that needs to perform this operation M times. With an array, the time complexity of this algorithm would be O(NM); with a hash set, only O(M). 2. Frequency A hash map can be used to translate keys to integers. Each integer represents the occurrences of the corresponding key. For example, suppose you want to verify if all characters in a string have the same frequency. A hash map is the ideal data structure for counting character frequencies in linear time. You can check if the frequencies are unique with a single iteration over the map.

Dangerous failures are not the ones that crash your app. They are the silent ones. Empty catch blocks, unawaited tasks, and exceptions printed to standard output are all different ways to lose exactly what you need when something breaks. Good exception handling is not about catching everything. It is about never losing information about a failure. Full breakdown in the latest issue of @EngPolymathic

HTTP client-side caching (explained simply in 3 minutes): HTTP servers host two types of resources: • Static → don’t change between requests • Dynamic → generated on the fly Since static resources don’t change, they can be cached to reduce load on the servers and lower response time for users. HTTP allows caching these resources on the client side using specific headers. Here’s how it works: 1. The client requests a GET for a resource that nobody accessed before. The local cache intercepts the request. Since the resource is not there, the cache requests it from the origin server. 2. The server tells the client that the resource can be cached using response headers: - Cache-Control → how long the resource can be cached - ETag → identifies a specific version of the resource 3. When the client receives the response, it caches the resource. 4. Later, when the client makes the same GET request: If the resource is still valid, the local cache returns it immediately. The server can still update a resource before it expires, so cache and server are not strongly consistent. This is usually an acceptable trade-off. The server can also force clients to fetch a new version by changing the ETag. If the resource has expired, the client sends a conditional request with If-None-Match (the ETag it has stored). - If the server has a new version → it returns it - If not → it replies with 304 Not Modified, and the cache renews the freshness of the stored resource You can also use Last-Modified + If-Modified-Since instead of ETag.

Most software engineers focus on how to read data from a cache. But reading is only half of the battle. Here are the 3 main cache writing strategies: 1. Write-through The application writes to the cache, and the cache immediately writes to the database. This keeps cache and database consistent. No data loss if the cache crashes. The downside is higher write latency. 2. Write-back The application writes to the cache. The cache writes to the database asynchronously. Better for write-heavy workloads: lower latency, less load on the database, more tolerant to database failures. The risk is data loss if the cache crashes before the data is flushed. 3. Write-around The application writes directly to the database. Data only enters the cache when it is read. Good for data that is written once and rarely read (for example, real-time logs). The downside is higher latency when reading recently written data (always a cache miss). The best strategy depends on the use case: write-back for write-heavy workloads, write-around when data is written once and rarely read.

I just published a new article on logistic regression. It is one of the most used classification models. If you have not used it, it returns a probability instead of a simple yes/no (spam 88%, not spam 12%). It builds on the perceptron concept from another article and shows how the two are nearly identical. The whole difference is one function at the end: replace the step function with a sigmoid, and “yes” turns into “yes, 88% sure.” The rest of the article covers the log loss, training, and softmax. Give it a read in this week’s issue of @EngPolymathic

How Amazon DynamoDB works under the hood (in under 3 minutes): 1. Data model DynamoDB has a flexible data model where data is saved in tables that hold items. Each item is made of attributes like numbers, strings, boolean, binaries, lists, maps, or sets. Because of this, you can model complex data relationships. Items are identified based on the partition key and the sort key. The partition key is mandatory, while the sort key is optional. You can use both keys to query data. For additional attributes, you need to create secondary indexes. 2. Scalability DynamoDB has almost infinite scalability. This is done by spreading tables across several storage nodes. The partition key is hashed to find the node for each item. The mapping between items and storage nodes can change dynamically. Partitions that get a lot of requests are moved to storage nodes with more power. 3. Availability DynamoDB stores each item on 3 nodes in different availability zones for redundancy. The nodes are kept in sync using a Paxos-based algorithm for consensus and leader election. You can always get to your data because DynamoDB saves the B-tree and write-ahead logs for each node in AWS S3.When a node fails, the leader copies the failed node’s B-Tree and Write-Ahead log to add a new node right away. Checksums are used a lot to make sure that data is correct. DynamoDB has many strengths but isn’t perfect though. First, a monolithic database might provide better performance for small data sizes. Second, you need a good understanding of your access patterns to design tables and effectively get most of the benefits.

Disjoint Set (Union-Find) shows up everywhere: Kruskal's algorithm, connected components, clustering, image segmentation, percolation, and more. Almost every developer has used an algorithm built on this idea. Only a few know how it actually becomes fast: - Quick-find stores each item’s representative directly, so checking two items is a single array read. The catch is that every merge has to rescan the whole array to relabel a group. - Quick-union stores each item’s parent instead. Merging becomes one assignment, but the trees can grow into a long chain, and lookups walk all the way up. - Union by size always hangs the smaller tree under the bigger one. An item only gets deeper when its tree at least doubles, which caps the height at log n. - Path compression repoints every item you pass on the way to the root, so the structure flattens the more you use it. Together they bring the amortized cost down to the inverse Ackermann function, which stays below 5 for any input you could realistically store. Even if you have used union-find in an interview, do you know why hanging the smaller tree under the bigger one is enough to cap every tree at log n? Full breakdown in the latest issue of @EngPolymathic

If you want a real mental model of how the internet moves your data, start with the 20-byte IPv4 header (most engineers have never looked inside): • TTL: stops packets from looping forever (decremented by every router) • Protocol field: what holds the layers together (6 = TCP, 17 = UDP) • Identifier + flags + offset: how routers break up packets that are too big • Header checksum: must be recomputed at every hop because TTL changes TCP and IP both have checksums, but they protect different bits. Then IPv6 delivered the verdict: fragmentation, header checksum, and options are gone. Fixed 40-byte header, addresses 4× larger. Even if you use IP every day, do you know why routers never reassemble fragments and leave that job to the end hosts? Full breakdown in the latest issue of @EngPolymathic

Distributed systems fail in ways a single machine never does. Here are 12 articles that will help you to actually understand how they work (one topic per week is a solid plan): ↓

IPv4 has only about 4.29 billion addresses, and the world needed far more years ago. Yet the Internet still works. Every developer relies on the tricks that made this possible, but few really understand them: - Classful addressing divided the space into fixed /8, /16, and /24 blocks. Simple but extremely wasteful. - CIDR lets you split network and host anywhere - Private addresses can be reused everywhere (the catch is that they don't work on the Internet) - NAT rewrites private addresses and port numbers so that thousands of devices can hide behind a single public address - DHCP hands out addresses automatically The key distinction is that your 192.168.x.x address only has meaning within your network. The single public address of your router is what the rest of the Internet sees. Even if you know this, do you actually know how a reply from a web server finds its way back to the right device behind your router? Full breakdown in the latest issue of @EngPolymathic

Most performance problems come down to how you use memory. And this paper is still the best place to understand it: • How RAM actually works • CPU caches (and why they make or break performance) • Practical optimization techniques • Tools to measure what's really happening Written by Ulrich Drepper (Red Hat). Almost 20 years old and still holds up. 100% free: people.freebsd.org/~lstewart…

There's a difference between knowing how a system works and understanding why it was built that way. The first you can get from a diagram in 30 seconds. The second takes real time, and it's the only one that lasts. Most technical content today sells you the first one. You read that a database uses an LSM tree, nod, and move on. Two weeks later it's gone, because you never learned what problem the LSM tree was solving in the first place. The engineers I trust most learned it the slow way. They didn't remember that writes go to a log first. They knew that random writes are bad for disks, so they need to be converted into sequential writes and paid later on through compaction. Now they can trace the entire design back to that one trade-off, years later. That's the difference. Facts you memorize decay. A tradeoff you understand stays with you and transfers. The moment you hit a new system, you already know which questions to ask. It's slower to learn this way. It's also the only way I know that compounds. That's why I write @EngPolymathic the way I do. From the ground up, tradeoffs first, no shortcuts.

A few algorithms I regularly force myself to review so I don’t forget how they work: • Kadane’s → Maximum subarray sum • Rabin-Karp → Substring search with hashing • Topological Sort → Ordering a DAG • Prim’s → Minimum Spanning Tree • Kruskal’s → MST with Union-Find • Dijkstra’s → Shortest path (no negative weights) • Bellman-Ford → Shortest path (with negative weights) • Tarjan’s → Strongly Connected Components • Backtracking → Sudoku Solver These are the ones I keep coming back to years later. Which algorithms do you still periodically revisit?

The fastest way to slow down your whole team is to become a bottleneck. A lot of people do it without even realizing it. It's rarely dramatic. It shows up in the small delays. The longer you take to answer a question, the longer a decision sits in limbo. The slower you review a pull request, the longer your teammate's work waits to ship. Each one seems small. They don't stay inside, though. A slow review stops a merge, which stops a deploy, which stops the next person who was waiting on it and so on. The wait affects everyone further down the line. Here's the part that matters for you. This unblocking work is exactly the work people put off, because it's not the exciting part and it doesn't feel like "your" task. Answering the question, reviewing the PR, making the decision so others can move. That's precisely why doing it well sets you apart. When you make everyone around you faster, you stop being just another contributor and become the person the team can't run without. And that, far more than your own output, is what actually moves your career forward.

System design has no right answers, only trade-offs. Here are 15 of the most common ones every engineer should know: 1. Scalability vs performance (handle more work vs do each task faster) 2. Vertical vs horizontal scaling (more powerful hardware vs more servers) 3. Latency vs throughput (fast individual responses vs more tasks over time) 4. Monolith vs microservices (a single unified application vs many small independent services) 5. Stateful vs stateless architecture (remember user sessions vs treat each request as independent) 6. Synchronous vs asynchronous processing (wait for each task to finish vs move on and handle results later) 7. Batch vs stream processing (process data in chunks vs handle it as it arrives) 8. SQL vs NoSQL (structured data and strict schemas vs flexibility for varied data) 9. Normalization vs denormalization (no duplicated data vs faster reads with some redundancy) 10. Consistency vs availability (up-to-date data everywhere vs a system that's always up) 11. Strong vs eventual consistency (immediate accuracy on all nodes vs a short delay in exchange for availability) 12. Read-through vs write-through cache (cache data when it's requested vs write to cache and database together) 13. TCP vs UDP (reliable and ordered delivery vs faster but possibly unordered) 14. Long polling vs WebSockets (a request that waits for data vs a continuous two-way connection) 15. REST vs GraphQL (standard HTTP methods vs querying exactly what you need) The real skill isn't picking a side. It's knowing exactly what you're giving up with every choice. Which trade-off are you struggling with the most right now?

Every packet on the Internet needs an address, but few developers really understand how IP addressing works. An IPv4 address has two parts: - Network portion → which network? - Host portion → which device on it? The tricky part is the line between them, and that line is drawn by the subnet mask. Even if you read addresses like 192.168.1.100 every day, do you know how a device figures out which network it belongs to? Or why the mask exists at all? A few key pieces: - An IPv4 address is really a 32-bit number split into four octets. Dotted-decimal (192.168.1.100) is just for our convenience, and each octet only goes from 0 to 255. - The subnet mask marks where the network part ends and the host part begins. It follows one strict rule: a block of 1s followed by a block of 0s, with no mixing, which is why values like 100 can never appear in a mask. - A bitwise AND between the address and the mask gives the network address. It's the exact operation a device runs, but you can also do it by hand once the mask lands on an octet boundary. - Every network reserves two addresses: the network address (all 0s in the host part) and the broadcast address (all 1s). These don't always end in .0 and .255, which trips a lot of people up when the mask sits mid-octet. - Destination addresses come in three flavors: unicast (one device), multicast (a group that opted in), and broadcast (everyone on the network). Routers don't forward broadcasts by default. - A device can only talk directly to others on the same subnet. So before sending, it runs that AND to decide "local or remote?" and hands anything remote to its default gateway. I just published a full breakdown (with binary examples) of how IPv4 addressing and subnetting actually work. Link to the latest issue of the @EngPolymathic below:

Most engineers learn system design by memorizing diagrams for the interview. The moment the question changes, it all falls apart. Here are the 12 topics I wish I had learned first to truly understand how large systems work, one per week (bookmark this): ↓