Using replicas to scale out your database
High traffic websites often use database replicas to scale out their reads. Most web traffic is a GET request anyway and never modifies data.
But how do you know when you should READ from the primary?
The answer is more complex than you might think. And it's important to understand the details.
Whenever data is updated on the primary, the change needs to be replicated to each of the replicas. The time this takes is known as “replication lag”.
You need to understand this concept to be able to work with replicas effectively.
Healthy replication lag is usually just a few milliseconds. But if you have a busy database, or are maybe running a schema change, it can grow to seconds or even minutes.
Imagine this scenario. If you create a pull request on GitHub. That’s a WRITE. Then you get redirected to your pull request. If the next page reads from a replica and it’s even 1 second behind, the page will 404 because the data doesn’t exist on the replica yet.
How to solve this: Websites like GitHub intelligently use the primary or replica based on what you’re doing. Most requests will go to replicas. But anytime you update data, GitHub will direct all your requests to hit the primary for a short period of time. This is called “automatic role switching”.
It solves the problem of sending and update and then immediately reading it back. This is known as "read your own writes".
GitHub has had this functionality for years and even contributed it back to Rails.
If your framework doesn’t make this easy for you. You’re probably not GitHub anyway and don’t need to auto swap connections. You can instead start scaling out your reads by only putting queries where it’s acceptable if the data is a little stale. Think analytical queries, dashboards, logged out user pages. These queries are usually the most expensive anyway, so it’s a performance win removing the load from your primary.
Learn more
My friend and co-worker Ben has a great (free!) video course on database scaling that teaches replication. Database scaling: Replication. Then if you're using Rails, well, all of this made pretty easy. You just need to be sure to understand replication lag.