System Design has its own language.
And just like with any language, it requires a specific terminology for people to communicate ideas with each other.
This is where knowledge of core concepts of System Design becomes extremely useful.
In this post, I will walkthrough the concepts and explain what they are:
Scalability
Scalability is the biggest magic word that’s thrown around in many system design discussions.
If someone comes to you and says that system ABC is not scalable, the assumption is that it’s bad.
The vice-versa is also true. If system XYZ is scalable, the assumption is that the system is good.
But what does scalability really mean?
Here’s what I think quite strongly:
Scalability is the ability of a system to handle an increased workload by repeatedly applying a cost-effective strategy for extending a system’s capacity.
Let’s say you run a restaurant and are committed to giving good service to the customers. You want to keep the same level of service no matter how many customers turn up.
How do you scale your service when the number of customers increases from 10 to 50? Or from 50 to 100? Or from 100 to 500?
For example, if waiters become a bottleneck, would it help by adding more waiters? If the cook’s bandwidth is an issue, would it help if we bring another cook to share the workload?
What if at some point adding more waiters and cooks is no longer cost-effective?
That’s the point when it’s no longer possible to scale your restaurant anymore.
The point is that there’s no infinite scalability. At some point, scaling is no longer cost-effective.
Here’s an illustration that shows this situation.
In my view, this is one of the most fruitful ways of looking at scalability.
However, there are other views as well.
Here’s one from a nice blog post from 2006.
A system is said to be scalable if when we increase the resource in a system, it results in increased performance in a manner proportional to the resources added.
Of course, adding resources is not just for scaling a system. In a distributed architecture, you also add more resources to improve reliability and introduce redundancy for your critical systems.
Either way, scalability is an extremely important concept in system design and you cannot design a scalable system by considering it as an afterthought.
By the way, if you are interested in learning about System Design concepts, you’d love the Progressive Coder Newsletter where I explain such concepts in a fun and interesting approach.
Subscribe now and join along.
Latency and Throughput
Let’s say Tom goes to a restaurant and orders a cheese sandwich (duh, a restaurant once more!)
Anyways, the waiter takes the order and returns with the sandwich after 10 minutes.
That’s the latency (basically, the amount of time it takes for a system to respond to a request)
What about throughput?
Imagine Tom, Jack, and Darcy go to the restaurant and give their order (cheese sandwiches) at the exact same time (they use quantum clocks). The restaurant was able to take all 3 orders and start processing them.
That’s the throughput (basically, the number of requests that a system can handle at the same time).
So, what should a system designer aim for?
Try and shoot for maximal throughput with acceptable latency.
More throughput can impact latency. For example, if the restaurant takes too many orders simultaneously but it increases their time to deliver the item on the table, the latency for a single order has gone up.
If the throughput is kept as less as possible, the number of customers a restaurant can serve at a time goes down.
As you can notice, it’s a game of trade-offs. And while designing a system, these things should be kept in mind.
Availability and Consistency
The restaurant has 2 chefs and one of them goes on leave for a day.
Can the restaurant operate without the missing chef?
If yes, you can say the restaurant has good availability.
In the context of a software system, availability is a measure of the system’s ability to provide services to its clients in the presence of failures.
But how do you measure availability?
In software systems, we use the concept of nines.
Three nines (99.9%) are mostly considered acceptable for a large number of systems. Anything above four nines (99.99%) is highly available. An even higher number of nines may be needed for mission-critical systems.
Here’s a handy chart that demonstrates the availability percentages in terms of actual downtime.
But what about consistency?
Let’s say the restaurant maintains 3 copies of all the orders made by the customers.
- The waiter’s copy
- The cook’s copy
- The billing counter copy
Are the three copies always in sync?
Are there periods of time when the order list at the kitchen table has different information from the list at the billing counter?
These are questions where the answer depends on the consistency requirements of your application.
As with a lot of things, there is a trade-off between availability and consistency.
Systems that prioritize high availability may sacrifice consistency, while systems that prioritize consistency may sacrifice availability.
Conclusion
System Design is not an exact science.
And that’s what makes it so interesting.
Knowing about the core concepts of system design is the first step in understanding how to build a system. After that, you’ve to consider all of these concepts and their trade offs to actually build an ideal system for your requirement.
If you have any comments or queries about this post, feel free to mention them in the comments section below.
Anyways, before we end this post, a quick reminder about the Progressive Code Newsletter where I explain software concepts using a storytelling approach so that you never forget what you’ve learned. I’m 100% sure you’d love it.
Subscribe now and see you over there.
0 Comments