15 articles to help you get better at System Design: [1] 4 Challenges of Distributed Systems - And Possible Solutions newsletter.systemdesigncodex… [2] Your Code is Fast, But Your Database is Slow—Now What? newsletter.systemdesigncodex… [3] 15 Must-Know Elements of System Design newsletter.systemdesigncodex… [4] How To Do Well in System Design Interviews? newsletter.systemdesigncodex… [5] 8 Must Know Distributed System Design Patterns newsletter.systemdesigncodex… [6] 6 Strategies to Build Secure APIs newsletter.systemdesigncodex… [7] How HTTP 2 Improves Upon HTTP 1 newsletter.systemdesigncodex… [8] How Netflix Uses Caching to Hold Attention? newsletter.systemdesigncodex… [9] 8 Must-Know Strategies to Build Scalable Systems newsletter.systemdesigncodex… [10] 3 Interview Questions on Event-Driven Patterns newsletter.systemdesigncodex… [11] Don’t Use Sync Communication for Every Case newsletter.systemdesigncodex… [12] How Consistent Hashing Works? newsletter.systemdesigncodex… [13] The Journey of a SQL Query Through a Database newsletter.systemdesigncodex… [14] TCP 3-Way Handshake newsletter.systemdesigncodex… [15] Database Sharding newsletter.systemdesigncodex…

What’s the most important part of Event-Driven Architecture? Events, of course. In an EDA-based application, different components communicate by sending and receiving events. But what exactly is an event? It is an occurrence that has happened in the application. Some examples of events: - A customer signing up for your application - A payment is received for an order - A failed authentication attempt Whatever their nature may be, there are mainly 3 different patterns for handling events: - Event Notifications - Event-Based State Transfer - Event Sourcing ✅ Event Notifications In this approach, events are used to notify something has occurred within your application. Such an event carries the absolute minimum state. Most times, it’s just the identifier (ID) of an entity. When receiving such events, some components may record the information for auditing purposes. Other components might call the source component to fetch additional information about the event. Such events are general-purpose in nature and suitable for broadcasting requirements. Less data is sent over the network in this case. ✅ Event-Based State Transfer This is more like the asynchronous relative of REST. Instead of REST’s on-demand pull model, event-based state transfer follows a push approach. Data (id + information) is sent out to be consumed by any components. Some components may create their own local cached copies of the data. Others may take some other action based on the data received. The advantage of this approach is that the consumer need not call back the source component for additional information. The entire information is right there in the event payload. ✅ Event Sourcing This is not about event transmission but event storage. Changes to an entity are not captured as irreversible modifications but are stored as events in an event store. These changes can be read and processed to recreate the final state of an entity whenever required. 👉 So - which patterns for EDAs have you used?

What’s the deal with Stateless Architecture? The term “stateless architecture” can be a little confusing. It seems to imply that the system is without any state. This is not exactly true. Stateless architecture just means that the state information (if needed) is stored outside the application server. Let’s understand it via a real-life example: ✅ You are visiting a stateless restaurant and each waiter has a perfectly forgetful memory. ✅ They just note the order on a separate system using a handheld device and forget about it. ✅ You make an order but then realize, you have an allergy to a particular item and want to change the order. ✅ The waiter will simply check their device to get the details of the order and make changes to it as required. 👉 The same is the case with a system built using Stateless Architecture. In such a system HTTP requests from a client can be sent to any of the servers. The state is typically stored on a separate database or cache that is accessible by all the servers. This way the load balancer can distribute the incoming requests more or less evenly across the instances. 👉 So - do you build your applications in a stateless manner?

24 principles that can help you navigate System Design Interviews: ✅ If you need to scale an individual component, stick with Vertical Scaling first. Then, look at Horizontal Scaling when needed ✅ To deal with traffic spikes, evaluate the use of Autoscaling for Dynamic Resource Allocation ✅ If the system requires High Availability with Performance, consider using a Load Balancer ✅ If you’re building a read-heavy system, it’s a good idea to use a Cache ✅ Listen to the system requirements properly and note them down if needed ✅ To reduce the latency of static assets, consider using a CDN ✅ If you want to improve the database query performance, focus on the right indexes ✅ To scale the read requests to your database, consider Database Replication ✅ To scale write requests, consider Database Sharding ✅Clarify your assumptions before jumping into the solution ✅ For global distribution of data, use a CDN ✅ If you want to store complex data such as videos, images and files, go for an Object Storage ✅ To manage the load on the server and prevent denial-of-service attacks, use Rate Limiting ✅ In order to remove a Single Point of Failure, implement Redundancy and Isolation ✅ Clarify about the Non-Functional Requirements as well and take them into account ✅ To improve the Fault Tolerance and Durability of your data, consider Data Replication techniques ✅ If the system needs to execute long-running tasks, use Async Processing and Background Processes ✅ To build a loosely coupled system, consider the use of Event-Driven Architecture ✅ If the system needs to store unstructured data, consider NoSQL databases ✅ Keep getting feedback during the discussion rather than waiting till the end ✅ To deal with large amounts of data flowing through the network, use Compression and Pagination techniques ✅ For automated builds and deployments, consider implementing CI/CD pipelines ✅ If you want to achieve independent deployments of various parts of the system, consider microservices architecture ✅ Establish good eye contact with the interviewer and answer the questions confidently. Above all else, remember that System Design questions don't have perfect answers. They are meant to test your analytical skills and ability to consider trade-offs. 👉 So - what would you add to the list?

Layer 4 vs Layer 7 Load Balancers Simplified ✅ Layer 4 Load Balancer As the name suggests, Layer 4 load balancers operate in the transport layer of the OSI model. Yes, the famous OSI model! What does it mean? The routing decisions are made solely based on the info available at that layer. Information such as IP address or port. In other words, a layer 4 load balancer cannot see the data in the network packet. It has some advantages: - Simpler to run and maintain - Better performance since no data lookup - More secure since there’s no need to decrypt the TLS data - Only one TCP connection But there are disadvantages as well. - No smart load balancing - No routing to different service types - No caching since the LB can’t even see the data. ✅ Layer 7 Load Balancer Layer 7 load balancers operate at the layer 7 of the OSI model. They deal with protocols such as HTTP(S), WebSocket, FTP, and SMTP. What this means is that Layer 7 LBs can see the actual data within the packet and make routing decisions based on that data. Here’s how they work: [1] Let’s say you have two microservices - one dedicated to the blog posts (/blog) and the other dedicated to the comments (/comments). [2] The client makes a request to the load balancer by establishing a TCP connection and says that it wants to access the route /blog. [3] The load balancer decrypts the data and checks the configured rules [4] Based on the request destination, the load balancer establishes a new TCP connection to the server instances hosting the blog microservice. [5] Once the response comes back, it sends over the response to the client. Advantages: - Smart load balancing (big benefit) - LB can also cache data - Can play the role of a reverse proxy Disadvantages: - Expensive to run and maintain - Needs to decrypt the data - Maintains 2 TCP connections - one from the client to LB and another from LB to the service 👉 So - have you used Layer 4 or Layer 7 load balancer? Or have you used both together?

How do you scale a component? There are three points that I consider: ✅ Vertical Scaling vs Horizontal Scaling The easiest way to scale a component is vertical scaling. Think bigger RAM. A larger disk or CPU. It's cost-effective in the short term. Plus, you avoid dealing with distributed stuff. But it becomes expensive in the long run and doesn't help you get rid of single points of failure. This brings us to Horizontal scaling i.e. scaling a component by adding more machines. ✅ Horizontally Scaling Stateless Apps A stateless component doesn't save the client data from one session for use in the next session. This makes it easier to scale horizontally. You can use a load balancer to distribute requests between multiple stateless components. Of course, it becomes important to build the services in a stateless manner. ✅ Horizontally Scaling Stateful Components A stateful component saves data from one session to the next one. For example, a web application that stores the session info. Or a database/key-value store. For scaling out stateful components, we've to adopt techniques like replication, sharding, and so on. 👉 TL;DR - Vertical scaling is less complex and you should stay with it for as long as possible. It can also be sufficient for a majority of the apps. - Think horizontal scaling when vertical is no longer cost-effective. - Try to build your application layer in a stateless manner so that it's easy to scale out. - For stateful components like databases, choose modern options designed for horizontal scalability. It will make your life easier. 👉 So - how do you approach the task of scaling a component?

15 articles to help you get better at System Design: [1] 6 Strategies to Build Secure APIs newsletter.systemdesigncodex… [2] How HTTP 2 Improves Upon HTTP 1 newsletter.systemdesigncodex… [3] How Netflix Uses Caching to Hold Attention? newsletter.systemdesigncodex… [4] 8 Must-Know Strategies to Build Scalable Systems newsletter.systemdesigncodex… [5] 8 Must-Know Strategies to Build Scalable Systems newsletter.systemdesigncodex… [6] Don’t Use Sync Communication for Every Case newsletter.systemdesigncodex… [7] How Consistent Hashing Works? newsletter.systemdesigncodex… [8] The Journey of a SQL Query Through a Database newsletter.systemdesigncodex… [9] TCP 3-Way Handshake newsletter.systemdesigncodex… [10] Database Sharding newsletter.systemdesigncodex… [11] Eventual Consistency is Tricky newsletter.systemdesigncodex… [12] Intro to GraphQL newsletter.systemdesigncodex… [13] 3 Kafka Messaging Strategies newsletter.systemdesigncodex… [14] Intro to Kafka newsletter.systemdesigncodex… [15] 3 Interview Questions on Event-Driven Patterns newsletter.systemdesigncodex…

Load Balancer vs API Gateway vs Backend-For Frontend simplified. ✅ Load Balancer When you check in to a hotel, you are usually greeted by a receptionist. The receptionist checks your documents, enters your info into their system, and guides you to your room. If there are too many guests, the hotel might keep multiple receptionists. Load balancers do a similar job for your application. It’s a tool that distributes incoming requests across multiple servers. Also, load balancers keep your system available. If one server goes down, it directs the request to another server. ✅ API Gateway Imagine a scenario where a company is conducting interviews for various positions, such as Tech Architect, Engineering Manager, and Software Engineer. Each position has a dedicated interview panel. Now, you don’t want the candidates to check each room to find the correct panel. This is where a coordinator checks the candidate’s profile and directs them to the correct panel. For a system, API Gateway is that coordinator. API Gateways can also perform different functions like authentication, rate limiting, logging, and caching. ✅ Backend for Frontend Let’s take our interview example further. For every position (Arch, Engineering Manager, Software Engineer), there are specific rounds of discussions. Some rounds are different based on the position but some may be common for all positions (such as the Director Round). How do you coordinate this for each position? Backends for Frontends is your BFF in this scenario. It’s a coordinator that takes care of the specific rounds for a particular type of interview. In system terms, BFF is the component that can take care of different device requirements. 👉 So - have you used Load Balancers, API Gateways & BFFs?

4 must-know Microservice Design Patterns: ✅ Database per service - In this pattern, each microservice manages its own data. - This means that no other service can access that data directly. - Communication or exchange of data can only happen through the owner service. - The success of this pattern depends on how well you’ve defined the bounded context of your application. ✅ Shared Database - Avoid this pattern as much as possible - But sometimes, it’s the only viable option to incrementally move to a microservice architecture. - This approach is lenient in the sense that multiple services use a shared database. - However, it creates a bigger impact surface and chances of run-time issues. ✅ API Composition - This pattern tries to solve the problem of implementing complex queries in a microservices architecture. - An API Composer invokes other services in the required order. - After fetching the results, it performs an in-memory join of the data before returning it to the caller. - This is an inefficient approach due to in-memory joins on potentially large datasets. ✅ CQRS + Event Sourcing - Next, we have CQRS - It can help get around the issues with the API Composition Pattern. - An application listens to domain events from other services and updates a separate query database. This makes it easy to serve complex aggregation queries. - CQRS can also be combined with Event Sourcing where you store the state of the entity (aggregate) as a sequence of events. - Of course, this pattern can sometimes be unfamiliar to the developers. At the end of the day, no pattern is perfect. Learn about the trade-offs with each pattern and choose what works for you. 👉 So - which patterns have you used?

If you are working with databases, you should know about ACID. ACID in databases stands for Atomicity, Consistency, Isolation, and Durability. Here’s what each term means: ✅ Atomicity (A) Atomic means something that cannot be broken down. Atomicity describes what happens to a Transaction that’s executing multiple write operations and something goes wrong halfway. Think of a Transaction as a bridge between two states of data named A and B. No intermediate states are allowed. It’s all or nothing. So, if you can’t complete the entire transaction successfully, you don’t complete any part of it. and the transaction is completely aborted. Without atomicity, a mid-way error during multiple updates, the entire database state can turn into a mess. Retrying the transaction would be risky because it can result in duplicate data. Atomicity saves you from all this trouble. ✅Consistency (C) Consistency in ACID is all about the context of your application. In any application or system, some statements must be true. For example, credits and debits in a bank must always be balanced. A consistent transaction is one where statements are true at the beginning as well as the end of the transaction. But most of it depends on your application. A database cannot save you if you write bad data that violates the invariants. In a nutshell, Consistency is the property of your application. ✅ Isolation (I) Isolation is the trickiest guarantee. It’s a given that your database will be accessed by multiple clients at the same time. Databases can handle this quite well if they are reading and writing different parts of the database. But the plot thickens when different clients try to access the same database records. This ends up creating concurrency issues. An example of this issue is two clients simultaneously trying to increment a counter that is stored in a database. Despite two increment requests, the counter was only incremented by 1 because of race conditions. The goal of Isolation is that concurrently executing transactions are isolated from each other. But it’s more of a degree than an absolute value. There are multiple isolation levels ranging from weak to strong. Some common ones are: - Read Uncommitted - Read Committed - Repeatable Read - Serializable ✅ Durability (D) Lastly, the purpose of a database system is to let you store data without any fear of losing it. Durability is the guarantee that promises this safety. The guarantee holds even in case of a hardware fault or database crash. There are two ways databases achieve this: - Disk storage and write-ahead logs for single-node databases - Replication for multiple nodes. Of course, there is no perfect durability. It’s all about risk-reduction techniques. 👉 So - what would you add to make the concept of ACID even more clear?

Cookies vs Session Simplified A cookie is basically a key-value pair that’s stored on the browser. How does it work? [1] The user logs in to your frontend application. [2] The frontend sends the request to the backend server [3] The backend server generates a cookie [4] It sets the cookie on the browser via the Set-Cookie response header. [5] The user makes a new request to view a different page. [6] The front end sends the request to the backend and includes the Cookie as part of the header. [7] The server checks the cookie for the user and responds with the required data. Sounds good, doesn’t it? But there’s one issue with using cookies. Cookies are accessible via the browser. You can modify the cookie information. That’s why it’s not a good idea to use cookies for storing sensitive data about the users. 👉 This is where sessions come into the picture. The session contains a unique set of characters to identify the user. It works as follows (the animation also explains the steps): [1] The user makes a login request [2] The frontend sends the request to the backend server [3] The backend creates a session using a secret key and stores it in some sort of session storage (database or cache) [4] Next, the server sends a cookie back to the client [5] However, the cookie contains the unique identifier for the session [6] The user makes a new request to view another page. [7] The browser sends the session ID as part of the cookie. This time only the server can validate whether the session is valid. A few important points to note over here: ✅ Cookies can have a “Secure” flag indicating that it should only be sent over HTTPS. This is good for security reasons. ✅ Also, “HttpOnly” cookies restrict the cookie’s access to JavaScript reducing the risk of XSS attacks. ✅ Cookies (especially 3rd party cookies) raise a bunch of privacy concerns because they can be used to track user behavior. ✅ While cookies can be made secure, server-side sessions provide additional layers of security against CSRF attacks and handling sensitive information ✅ Also, server-side sessions can be centrally managed. This means you can invalidate sessions, expire, or revoke them if needed. 👉 So - what do you typically use for authentication?

Database Caching 101 (5 strategies you must know) 👉 Cache-Aside Strategy In this strategy, the cache sits next to the database. Here’s how it works: - When there is a request for data, the application first checks the cache - If there’s a cache hit, the application returns from the cache - In case of a cache miss, the application queries the DB and returns the data - The application also stores the missing data in the cache for future requests Pros: Great for read-heavy workloads. Also, better resiliency since a cache failure cannot cripple the system. Cons: Potential inconsistency between the cache and the database. 👉 Read-Through Strategy The cache sits between the application and the database. Here’s what happens in read-through: The application goes to the cache for any read request. If there’s a cache hit, data is returned from the cache and that’s the end of the flow. In case of a cache miss, the cache gets the missing data from the database and returns it to the application. Pros: The application doesn’t have to worry about fetching from the database or the cache. The cache takes care of it. Cons: Potential inconsistency between the cache and the DB and the need to go to the database for every brand-new read request. 👉 Write-Around Strategy Same as Cache-Aside with the added context about the write operations. In this strategy, all writes go to the database and the data that is read goes to the cache. For a cache miss, the application reads from the DB and updates the cache for the next time. Great for cases where data is only written once and rarely updated (like a blog post or a static website). 👉 Write-Through Strategy Write-through tries to solve the problems with read-through. Instead of writing to the DB, the application first writes to the cache. And the cache immediately writes to the DB. The word “immediately” is the key over here. Pros: Cache will always have any written data. New read requests won’t experience a delay while the cache requests the data from the main DB. Cons: Extra write latency because the data must go to the cache and then to the DB. 👉 Write-Back Strategy It’s a variation of the write-through strategy. With one key difference… In the write-back, the application writes directly to the cache. However, the cache doesn’t immediately write to the database but after a delay. Pros: The strain on the cache is reduced in case you have a write-heavy workload. Requests to the DB are batched and the overall write performance is improved. Cons: In case of a cache failure, there are chances of possible data loss. 👉 So - which strategy have you used?

Don't go for caching without asking these 7 questions: ✅ Am I using the cache to hide the real issues? Inefficient data models and improper indexes might be the real issues ✅ Is the data I'm trying to cache frequently accessed? It's all about considering the ROI of caching ✅ Is the data I'm trying to cache dynamic? There is not much advantage in caching an item that's too dynamic. ✅ Which caching strategy should I choose? Cache-aside is popular. But there are also other strategies that might be worth exploring. ✅ How will I maintain cache consistency? No one likes caches with stale data. ✅ Am I ok with a cold cache or do I need a warm cache? Warm cache requires much more investment in engineering and design ✅ How will I measure the cache effectiveness? Without measurements, you won't know whether it's even useful. 👉 So - what other question will you ask?

What you must know about Kafka? 👉 What is Apache Kafka? Apache Kafka is a distributed event store and streaming platform. It began as an internal project at LinkedIn. Over time, it grew rapidly and today, some of the largest data pipelines in the world use Kafka. Organizations like Netflix and Uber rely on it for their workflows. 👉 Kafka Messages, Topics, and Partitions The basic unit of data in Kafka is a Message Think of a message like a record in a database table. It is transmitted as an array of bytes. Every message goes to a particular Topic. You can compare Kafka Topics to a database table or a folder on your computer. Topics are also made up of multiple partitions. Partitions improve the redundancy and make the topics horizontally scalable. 👉 Kafka Producers and Kafka Consumer Producers in Kafka create new messages, batch them, and send them over to a Kafka topic. A producer also balances messages across the different partitions of a topic. You can provide a custom partitioning strategy to control the distribution of messages. Kafka Consumers read messages from a broker. One or more consumers work as a consumer group to consume messages from a topic. A consumer instance is tied to a particular partition. In other words, a partition is owned by a consumer instance. 👉 Kafka Broker and Cluster A single Kafka server is known as a Broker. A broker can handle thousands of partitions and millions of messages/second. Think of the broker as a bridge between the producer and consumer. It receives messages from producers and handles fetch requests from the consumer. But the broker is part of a Kafka Cluster A Kafka Cluster consists of several brokers and provides features like replication. Every partition is replicated across multiple brokers ensuring high availability and redundancy. 👉 So - have you used Kafka?

I asked 11 developers and found that 8 were struggling with data consistency issues. The number one culprit was Dual Writes. Dual Writes happen when you have to update two different systems. For example, [1] Updating a database [2] Publishing an event to a different system (say Kafka). The 2nd step could also be anything else like sending an email. Because these two systems aren’t linked, we can’t update both in a transactional manner. If the DB update is successful but a failure occurs after that, the event will never be published. This means inconsistent data. What’s the solution? 👉 The Transactional Outbox Pattern. In this pattern, the transactional logic is pushed into the database. Whenever there is an update in the database, we also update an outbox table in the same transaction. Think of the outbox table as a mailbox. As the database updates happen, it is filled with letters that have to be delivered to a post office. From an application point of view, - Letters = Events - Post Office = Kafka All we need now is the postman, who can carry those letters from the mailbox (outbox) to the post office (Kafka). This could be an async process and you’ve multiple options on how to implement it: - A separate thread with the original microservice - A separate application - Kafka Connector or Change Data Capture process that monitors the outbox table. 👉 But is the Outbox pattern perfect? Nothing is perfect, as such. With this pattern, you can still have duplicate messages in case there are failures. This is to ensure that we have an At-Least-Once delivery guarantee. But this requires the downstream systems to de-duplicate the messages. 👉 So - have you used the Outbox pattern?