Interview Preparation | Back of the Envelope Calculations
December 15th, 2023
Introduction
This summary will serve to further cement my learnings taken when reviewing the Back-of-the-Envelope Calculations module in the Grokking the Modern System Design Interview course, and I hope will provide some learnings to you as well.
Types of Data Center Servers
Web Servers
Web servers, separated from application servers for scalability, manage API calls and interact with clients. They require moderate computational resources, with examples like Facebook using web servers with 32 GB RAM and 500 GB storage.
Application Servers
Responsible for core application software, application servers provide dynamic content, demanding extensive computational and storage resources. Facebook’s application servers, for instance, feature up to 256 GB RAM and various storage types.
Storage Servers
As data volume surges, storage servers become crucial. YouTube, for instance, utilizes various datastores, including Blob storage and specialized options like Bigtable. Facebook’s servers, with up to 120 TB storage capacity, house exabytes of data.
Typical Server Specifications
Modern servers vary in specifications, with a typical server having two sockets, an Intel Xeon X2686 processor, 36 cores (72 hardware threads), 256 GB RAM, 45 MB L3 cache, and a 15 TB storage capacity. These figures serve as reference points, with more powerful machines supporting higher specifications.
Important Latencies and Rates
Below is a table that includes important latencies for different components in a system:
| Component | Time (nanoseconds) |
|---|---|
| L1 cache reference | 0.9 |
| L2 cache reference | 2.8 |
| L3 cache reference | 12.9 |
| Main memory reference | 100 |
| Compress 1KB with Snzip | 3,000 (3 microseconds) |
| Read 1 MB sequentially from memory | 9,000 (9 microseconds) |
| Read 1 MB sequentially from SSD | 200,000 (200 microseconds) |
| Round trip within the same datacenter | 500,000 (500 microseconds) |
| Read 1 MB sequentially from SSD with speed ~1GB/sec SSD | 1,000,000 (1 milliseconds) |
| Disk seek | 4,000,000 (4 milliseconds) |
| Read 1 MB sequentially from disk | 2,000,000 (2 milliseconds) |
| Send packet SF->NYC | 71,000,000 (71 milliseconds) |
Requests Estimation
Estimating the number of requests involves considering CPU-bound and memory-bound requests. Assumptions about server specifications, RAM usage, and task completion times play a role. For instance, a CPU-bound request calculation determines RPS based on CPU threads and task time.
CPU-Bound Requests:
Estimating requests bound by CPU involves determining the Requests Per Second (RPS) based on the computational capabilities of the server. The formula used is:
RPSCPU=NumCPU/Tasktime
Where:
RPSCPU​ is the CPU-bound Requests Per Second.
NumCPU​ is the number of CPU threads or hardware threads.
Tasktime​ is the time each CPU-bound task takes to complete.
For example, if a server has 72 CPU threads and each CPU-bound task takes 200 milliseconds, the calculation would be RPSCPU=72/0.2=360 RPS.
This calculation visualizes one second as a box and determines how many CPU-bound tasks can fit inside that box. More CPU threads result in a higher RPS.
Memory-Bound Requests:
Estimating requests bound by memory involves considering the server’s RAM size and the time it takes for memory-bound tasks to complete. The formula used is:
RPSmemory=(RAMsize/Workermemory)Ă—(1/Tasktime)
Where:
RPSmemory​ is the Memory-bound Requests Per Second.
RAMsize​ is the total size of RAM.
Workermemory​ is the amount of RAM consumed by a worker managing a request.
Tasktime​ is the time each memory-bound task takes to complete.
For instance, if a server has 240 GB of RAM, each worker consumes 300 MB of RAM, and each memory-bound task takes 50 milliseconds, the calculation would be RPSmemory=(240)/(.3Ă—0.05)=16,000 RPS.
This calculation assesses how many memory-bound processes a server can host and how many tasks can fit in each memory-bound process, providing insights into the server’s memory-bound request handling capacity.
Examples of Resource Estimation
Number of Servers Required
Assumptions
- 500 million daily active users (DAU)
- 20 requests per day per user
- Each server can handle 8,000 requests per second (RPS)
Approximation Equation
Number of servers (peak-load) = Number of daily active users/RPS of a server
Storage Requirements
Assumptions
- Twitter as a design example
- 250 million DAU
- Each user posts three tweets per day
- 10% of tweets contain images, 5% contain videos
- Image size: 200 KB, Video size: 3 MB
- Tweet text and metadata require 250 Bytes of storage
Storage Space Calculation
Total tweets per day = (250 million users)*(3 tweets)
Total tweet text and metadata storage = (Total tweets per day)*(250B)
Total Image Data Stored Per day = (Total tweets per day)*(.1)*(200KB)
Total Video Data stored per day = (Total tweets per day)*(0.05)*(3 MB)
Total storage required for one day = (Total tweet text and metadata storage) + (Total Image Data Stored Per day) + (Total Video Data stored per day)
Total storage required for one year = 365*(Total storage required for one day)
Bandwidth Requirements
Assumptions
- Using Twitter as an example again
- 128 TBs of storage required per day
- A user views 50 tweets per day.
Incoming Traffic Calculation
Incoming traffic bandwidth = Total storage required for one day/Number of seconds in a day = (128TB / 86400)*(1000 GB/TB)*8(Gb/GB) = ~12 Gbps
Outgoing Traffic Example
Number of Tweets in a Day = (500M)*50 tweets
Text and Metadata Data = (Number of tweets in a day)*250Bytes
Image Data = (Number of tweets in a day)*(.1)*200 kB
Video Data = (Number of tweets in a day)*(.05)*3 MB
Total Sent Data in a Day = Text and Metadata Data + Image Data + Video Data
Outgoing Traffic Bandwidth = Total Sent Data in a day / Number of seconds in a day
Total bandwidth requirements = Incoming traffic bandwidth + Outgoing traffic bandwidth