Database Sharding with Hands-on Labs

This is a beginner-friendly course on database sharding. It’s about understanding why sharding is needed, how it works, and when you should not do it at all, without memorizing terms.

The whole course follows one app called Chirp, a small social app where people post short messages, follow each other, and like posts. It starts on one database with a couple hundred users and grows to forty million users and eight terabytes of data. Every lesson starts with something breaking. A profile that used to open in thirty milliseconds now takes three seconds, writes pile up during a big event, a pop star with forty million followers overloads one machine, or adding a single machine forces eighty percent of the data to move. Then we fix it, and look at the new problem the fix creates.

Across seven lessons, we cover reads and writes, replicas, and why no replica can raise the write limit. We also cover what a shard is, how to choose the shard key, scatter-gather and co-location, hashing, and the four ways to split data. After that come the routing layer, hotspots and the celebrity problem, resharding, consistent hashing and virtual nodes, fixed buckets, and joins across shards. Then we look at how real databases like Vitess, Citus, Cassandra, DynamoDB and MongoDB do the sharding for you. We end on the three signals that tell you it’s actually time to shard. Everything is explained in plain language, with a hand-drawn diagram for every step.

We also have four hands-on labs on a real Citus cluster. In them, you split a users table across machines, test different shard keys and see which ones spread the data evenly, overload one shard with a pop star’s traffic and fix it, and finally add a new machine, move data onto it, and join tables that live on different shards. Every lab comes with hints and full solutions.

If you know what a database table is and can use a terminal, you’re ready to start.