AdvMaple

Uber has one of the largest Kafka deployments in the WORLD… So how do they secure it? mTLS and strong authorization rules. Uber models its production environment as a zero-trust network. Since any host can be compromised, they rely on strong cryptographic primitives to establish trust between services. 🥷 uPKI is Uber’s identity platform, based on top of Spire. It’s responsible for issuing short-lived, auto-rotated cryptographic key pairs. Those pairs consist of an X.509-SVID, a private key and a trust bundle. 🔐 With this setup, the system automatically refreshes (rotates) its certificates once they’re near expiry. The Spire agent keeps a long-lived connection open to the Spire server and proactively generates a new cert when the time is right. 💡 This is key, because it allows Uber to issue very short-lived certificates which then bound the time a bad actor has to make use of any leaked credentials. mTLS gives Uber authentication, confidentiality via encryption, and data integrity. 👍 What’s missing is authorization now. Similar to checking in to a flight after they’ve verified your ID (you are who you say you are) - the next check is: ✋ are you actually allowed to board this flight? (do you have the ticket with the right permissions) For this, Uber has a service named Charter. Similar to AWS IAM, it’s a framework allowing you to specify who and what can access specific services and resources. Kafka has a pluggable authorization framework, where you can configure a custom Authorizer class to do the authz for you (`authorizer_class_name`). This authorizer gets called as part of any request flow. In Uber’s config, it’s called with a pair of actor, resource and operation. 🎬 actor - the entity that is the subject of the authorization decision (e.g you at an airport). Also called a KafkaPrincipal in Kafka. 🪨 resource - the resource upon which the authorization decision is made (e.g the flight) 🔧 operation - the operation performed on the resource (e.g boarding). Also called an ACLOperation in Kafka The Authorizer makes a remote RPC call to Charter to figure out the decision for this pair, and from then-on it caches the result. Simple, right?