面試系統設計_系統設計面試問題

面試系統設計

You may have heard the terms "Architecture" or "System Design." These come up a lot during developer job interviews – especially at big tech companies.

您可能已經聽說過“架構”或“系統設計”這兩個術語。在開發人員工作面試中，尤其是在大型科技公司中，這些問題很多。

This in-depth guide will help prepare you for the System Design interview, by teaching you basic software architecture concepts.

通過指導您基本的軟件體系結構概念，本深入指南將幫助您為系統設計面試做準備。

This is not an exhaustive treatment, since System Design is a vast topic. But if you're a junior or mid-level developer, this should give you a strong foundation.

這不是詳盡的處理方法，因為系統設計是一個廣泛的主題。但是，如果您是初級或中級開發人員，這應該為您奠定堅實的基礎。

From there, you can dig deeper with other resources. I've listed some of my favourite resources at the very bottom of this article. ?

從那里，您可以深入挖掘其他資源。我在本文的底部列出了一些我最喜歡的資源。

I've broken this guide into bite-sized chunks by topic and so I recommend you bookmark it. I've found spaced learning and repetition to be incredibly valuable tools to learn and retain information. And I've designed this guide to be chunked down into pieces that are easy to do spaced repetition with.

我已按主題將本指南分為幾小部分，因此建議您將其添加為書簽。我發現間隔學習和重復學習是學習和保留信息的極有價值的工具。而且我已將本指南設計為易于分解的小片段。

Section 1: Networks & Protocols (IP, DNS, HTTP, TCP etc)
第1部分：網絡和協議(IP，DNS，HTTP，TCP等)
Section 2: Storage, Latency & Throughput
第2節：存儲，延遲和吞吐量
Section 3: Availability
第3節：可用性
Section 4: Caching
第4節：緩存
Section 5: Proxies
第5節：代理
Section 6: Load Balancing
第6節：負載平衡
Section 7: Consistent Hashing
第7節：一致的哈希
Section 8: Databases
第8節：數據庫
Section 9: Leader Election
第9節：領導人選舉
Section 10: Polling, Streaming, Sockets
第10節：輪詢，流式傳輸，套接字
Section 11: Endpoint Protection
第11節：端點保護
Section 12: Messages & Pub-Sub
第12節：消息和發布訂閱
Section 13: Smaller Essentials
第13節：較小的必需品

Let's get started!

讓我們開始吧！

第1節：網絡和協議 (Section 1: Networks and Protocols)

"Protocols" is a fancy word that has a meaning in English totally independent of computer science. It means a system of rules and regulations that govern something. A kind of "official procedure" or "official way something must be done".

“協議”是一個花哨的單詞，在英語中的含義完全獨立于計算機科學。這意味著管理某些事物的規則和法規系統。一種“官方程序”或“必須完成某些事情的官方方式”。

For people to connect to machines and code that communicate with each other, they need a network over which such communication can take place. But the communication also needs some rules, structure, and agreed-upon procedures.

為了使人們連接到彼此通信的機器和代碼，他們需要一個可以在其上進行通信的網絡。但是溝通也需要一些規則，結構和商定的程序。

Thus, network protocols are protocols that govern how machines and software communicate over a given network. An example of a network is our beloved world wide web.

因此，網絡協議是控制機器和軟件如何通過給定網絡進行通信的協議。網絡的一個例子是我們鐘愛的萬維網。

You may have heard of the most common network protocols of the internet era - things like HTTP, TCP/IP etc. Let's break them down into basics.

您可能聽說過Internet時代最常見的網絡協議-例如HTTP，TCP / IP等。讓我們將它們分解為基礎。

IP- 互聯網協議 (IP - Internet Protocol)

Think of this as the fundamental layer of protocols. It is the basic protocol that instructs us on how almost all communication across internet networks must be implemented.

將此視為協議的基本層。這是基本協議，可指導我們如何實現幾乎所有跨Internet網絡的通信。

Messages over IP are often communicated in "packets", which are small bundles of information (2^16 bytes). Each packet has an essential structure made up of two components: the Header and the Data.

IP上的消息通常以“數據包”形式進行通信，這些信息包是一小束信息(2 ^ 16字節)。每個數據包都有一個由兩個部分組成的基本結構：標頭和數據。

The header contains "meta" data about the packet and its data. This metadata includes information such as the IP address of the source (where the packet comes from) and the destination IP address (destination of the packet). Clearly, this is fundamental to being able to send information from one point to another - you need the "from" and "to" addresses. ?

標頭包含有關數據包及其數據的“元”數據。該元數據包括諸如源(數據包來自何處)的IP地址和目標IP地址(數據包的目的地)之類的信息。顯然，這是能夠將信息從一個點發送到另一點的基礎-您需要“從”和“到”地址。

And an IP Address is a numeric label assigned to each device connected to a computer network that uses the Internet Protocol for communication. There are public and private IP addresses, and there are currently two versions. The new version is called IPv6 and is increasingly being adopted because IPv4 is running out of numerical addresses.

IP地址是分配給連接到使用Internet協議進行通信的計算機網絡的每個設備的數字標簽。有公用和專用IP地址，當前有兩個版本。新版本稱為IPv6，由于IPv4的數字地址用完了，因此越來越多地被采用。

The other protocols we will consider in this post are built on top of IP, ?just like your favorite software language has libraries and frameworks built on top of it. ?

我們將在本文中考慮的其他協議是基于IP構建的，就像您最喜歡的軟件語言具有基于IP構建的庫和框架一樣。

TCP- 傳輸控制協議 (TCP - Transmission Control Protocol)

TCP is a utility built on top of IP. As you may know from reading my posts, I firmly believe you need to understand why something was invented in order to truly understand what it does.

TCP是建立在IP之上的實用程序。正如您可以從閱讀我的文章知道，我堅信你需要了解為什么有些為了被發明真正了解它做什么。

TCP was created to solve a problem with IP. Data over IP is typically sent in multiple packets because each packet is fairly small (2^16 bytes). Multiple packets can result in (A) lost or dropped packets and (B) disordered packets, thus corrupting the transmitted data. ?TCP solves both of these by guaranteeing transmission of packets in an ordered way. ?

創建TCP是為了解決IP問題。 IP上的數據通常以多個數據包發送，因為每個數據包都非常小(2 ^ 16字節)。多個數據包可能導致(A)數據包丟失或丟失，以及(B)無序的數據包，從而破壞了傳輸的數據。 TCP通過保證以有序方式傳輸數據包來解決這兩種情況。

Being built on top of IP, the packet has a header called the TCP header in addition to the IP header. This TCP header contains information about the ordering of packets, and the number of packets and so on. This ensures that the data is reliably received at the other end. It is generally referred to as TCP/IP because it is built on top of IP.

數據包建立在IP之上，除IP報頭外，還具有稱為TCP報頭的報頭。此TCP頭包含有關數據包順序和數據包數量等信息。這樣可以確保在另一端可靠地接收數據。由于它建立在IP之上，因此通常稱為TCP / IP。

TCP needs to establish a connection between source and destination before it transmits the packets, and it does this via a "handshake". This connection itself is established using packets where the source informs the destination that it wants to open a connection, and the destination says OK, and then a connection is opened.

TCP需要先在源和目的地之間建立連接，然后再通過“握手”進行連接。此連接本身是使用數據包建立的，其中源將其通知目的地它想打開連接，并且目的地說“確定”，然后打開連接。

This, in effect, is what happens when a server "listens" at a port - just before it starts to listen there is a handshake, and then the connection is opened (listening starts). Similarly, one sends the other a message that it is about to close the connection, and that ends the connection. ?

實際上，這就是服務器在端口“偵聽”時發生的情況-在它開始偵聽之前，先握手，然后打開連接(偵聽開始)。類似地，一個向另一個發送一條消息，即將關閉連接，并結束連接。

HTTP- 超文本傳輸??協議 (HTTP - Hyper Text Transfer Protocol)

HTTP is a protocol that is an abstraction built on top of TCP/IP. It introduces a very important pattern called the request-response pattern, specifically for client-server interactions. ?

HTTP是一種基于TCP / IP的抽象協議。它引入了一個非常重要的模式，稱為請求-響應模式，專門用于客戶端-服務器交互。

A client is simply a machine or system that requests information, and a server is the machine or system that responds with information. A browser is a client, and a web-server is a server. ?When a server requests data from another server then the first server is also a client, and the second server is the server (I know, tautologies).

客戶端只是請求信息的機器或系統，而服務器是響應信息的機器或系統。瀏覽器是客戶端，而網絡服務器是服務器。當服務器從另一臺服務器請求數據時，第一臺服務器也是客戶端，第二臺服務器是服務器(我知道是重言式)。

So this request-response cycle has its own rules under HTTP and this standardizes how information is transmitted across the internet. ?

因此，此請求-響應周期在HTTP下具有其自己的規則，這標準化了如何通過Internet傳輸信息。

At this level of abstraction we typically don't need to worry too much about IP and TCP. However, in HTTP, requests and responses have headers and bodies too, and these contain data that can be set by the developer.

在這種抽象級別上，我們通常不需要太擔心IP和TCP。但是，在HTTP中，請求和響應也具有標頭和正文，并且標頭和正文包含開發人員可以設置的數據。

HTTP requests and responses can be thought of as messages with key-value pairs, very similar to objects in JavaScript and dictionaries in Python, but not the same.

HTTP請求和響應可以被視為具有鍵-值對的消息，與JavaScript中的對象和Python中的字典非常相似，但并不相同。

Below is an illustration of the content, and key-value pairs in HTTP request and response messages.

下圖說明了HTTP請求和響應消息中的內容和鍵值對。

HTTP also comes with some "verbs" or "methods" which are commands that give you an idea of what sort of operation is intended to be performed. For example, the common HTTP methods are "GET", "POST", "PUT", "DELETE" and "PATCH", but there are more. In the above picture, look for the HTTP verb in the start line.

HTTP還帶有一些“動詞”或“方法”，這些命令是使您了解打算執行哪種操作的命令。例如，常見的HTTP方法是“ GET”，“ POST”，“ PUT”，“ DELETE”和“ PATCH”，但還有更多方法。在上圖中，在起始行中查找HTTP動詞。

第2節：存儲，延遲和吞吐量 (Section 2: Storage, Latency & Throughput)

存儲 (Storage)

Storage is about holding information. Any app, system, or service that you program will need to store and retrieve data, and those are the two fundamental purposes of storage.

存儲是關于保存信息的。您編程的任何應用程序，系統或服務都需要存儲和檢索數據，而這是存儲的兩個基本目的。

But it's not just about storing data – it's also about fetching it. We use a database to achieve this. A database is a software layer that helps us store and retrieve data.

但這不只是存儲數據，還在于獲取數據。我們使用數據庫來實現這一目標。數據庫是幫助我們存儲和檢索數據的軟件層。

These two primary types of operations, storing and retrieving, are also variously called 'set, get', 'store, fetch', 'write, read' and so on. To interact with storage, you will need to go through the database, which acts as an intermediary for you to conduct these fundamental operations.

這兩種主要的操作類型(存儲和檢索)也分別稱為“設置，獲取”，“存儲，獲取”，“寫入，讀取”等。要與存儲進行交互，您將需要遍歷數據庫，該數據庫充當您執行這些基本操作的中介。

The word "storage" can sometimes fool us into thinking about it in physical terms. If I "store" my bike in the shed, I can expect it to be there when I next open the shed.

“存儲”一詞有時會使我們從物理角度上去思考它。如果我將自行車“存放”在棚子里，那么我下次打開棚子時可以期望它在那里。

But that doesn't always happen in the computing world. Storage can broadly be of two types: "Memory" storage and "Disk" storage. ?

但這并不總是在計算世界中發生。存儲大致可以分為兩種類型：“內存”存儲和“磁盤”存儲。

Of these two, the disk storage tends to be the more robust and "permanent" (not truly permanent, so we often use the word "persistent" storage instead). Disk storage is persistent storage. This means that when you save something to Disk, and turn the power off, or restart your server, that data will "persist". It won't be lost.

在這兩個磁盤中，磁盤存儲趨向于更健壯和“永久”(不是真正的永久，因此我們經常使用“永久”存儲一詞)。磁盤存儲是持久性存儲。這意味著，當您將某些內容保存到磁盤，然后關閉電源或重新啟動服務器時，該數據將“持續存在”。它不會丟失。

However, if you leave data in "Memory" then that usually gets wiped away when you shut down or restart, or otherwise lose power. ?The computer you use everyday has both these storage types. Your hard disk is "persistent" Disk storage, and your RAM is transient Memory storage.

但是，如果將數據保留在“內存”中，那么通常在關閉或重新啟動時這些數據會消失，否則會斷電。您每天使用的計算機都具有這兩種存儲類型。您的硬盤是“永久性”磁盤存儲，而您的RAM是暫時性的內存存儲。

On servers, if the data you're keeping track of is only useful during a session of that server, then it makes sense to keep it in Memory. This is much faster and less expensive than writing things to a persistent database.

在服務器上，如果您跟蹤的數據僅在該服務器的會話期間有用，則將其保留在內存中是有意義的。與將內容寫入持久數據庫相比，這要快得多且成本更低。

For example, a single session may mean when a user is logged in and using your site. After they log out, you may not need to hold on to bits of data that you collected during the session.

例如，一個會話可能意味著用戶登錄并使用您的網站。他們注銷后，您可能不需要保留在會話期間收集的部分數據。

But whatever you do want to hold on to (like shopping cart history) you will put in persistent Disk storage. That way you can access that data the next time the user logs in, and they will have a seamless experience.

但是，無論您想保留什么(例如購物車歷史記錄)，您都將放入永久性磁盤存儲中。這樣，您下次用戶登錄時就可以訪問該數據，他們將獲得無縫的體驗。

Ok, so this seems quite simple and basic, and it's meant to be. This is a primer. Storage can get very complex. If you take a look at the range of storage products and solutions your head will spin.

好的，這看起來非常簡單和基本，而且確實如此。這是入門。存儲會變得非常復雜。如果您看一下存儲產品和解決方案的范圍，您的頭會旋轉。

This is because different use-cases require different types of storage. They key to choosing the right storage types for your system depends on a lot of factors and the needs of your application, and how users interact with it. Other factors include:

這是因為不同的用例需要不同的存儲類型。它們為系統選擇正確的存儲類型的關鍵取決于許多因素，應用程序的需求以及用戶與之交互的方式。其他因素包括：

the shape (structure) of your data, or
數據的形狀(結構)，或
what sort of availability it needs (what level of downtime is OK for your storage), or
它需要什么樣的可用性(哪種水平的停機時間適合您的存儲)，或者
scalability (how fast do you need to read and write data, and will these reads and writes happen concurrently (simultaneously) or sequentially) etc, or
可伸縮性(您需要多快讀寫數據，這些讀寫將同時(同時)或順序進行)等，或
consistency - if you protect against downtime using distributed storage, then how consistent is the data across your stores?
一致性-如果您使用分布式存儲來防止停機，那么整個存儲中的數據的一致性如何？

These questions and the conclusions require you to consider your trade-offs carefully. Is consistency more important than speed? Do you need the database to service millions of operations per minute or only for nightly updates? ?I will be dealing with these concepts in sections later, so don't worry if you've no idea what they are.

這些問題和結論要求您仔細考慮您的取舍。一致性比速度重要嗎？您需要該數據庫每分鐘執行數百萬次操作還是僅用于每晚更新？我將在后面的部分中處理這些概念，所以如果您不知道它們是什么，請不要擔心。

潛伏 (Latency)

"Latency" and "Throughput" are terms you're going to hear a lot of as you start to get more experienced with designing systems to support the front end of your application. ?They are very fundamental to the experience and performance of your application and the system as a whole. ?There is often a tendency to use these terms in a broader sense than intended, or out of context, but let's fix that.

當您開始在設計系統以支持應用程序的前端方面獲得更多經驗時，您會經常聽到“延遲”和“吞吐量”這兩個術語。它們對于您的應用程序和整個系統的體驗和性能至關重要。通常傾向于以比預期更廣泛的含義使用這些術語，或者出于上下文考慮，但讓我們對其進行修復。

Latency is simply the measure of a duration. What duration? The duration for an action to complete something or produce a result. For example: for data to move from one place in the system to another. ?You may think of it as a lag, or just simply the time taken to complete an operation.

延遲只是持續時間的度量。什么時間？完成某件事或產生結果的動作的持續時間。例如：數據從系統中的一個位置移動到另一個位置。您可能會認為這是一個滯后，或者僅僅是完成操作所花費的時間。

The most commonly understood latency is the "round trip" network request - how long does it take for your front end website (client) to send a query to your server, and get a response back from the server. ?

最常見的延遲是“往返”網絡請求-前端網站(客戶端)向服務器發送查詢并從服務器返回響應需要多長時間。

When you're loading a site, you want this to be as fast and as smooth as possible. ?In other words you want low latency. ?Fast lookups means low latency. ?So finding a value in an array of elements is slower (higher latency, because you need to iterate over each element in the array to find the one you want) than finding a value in a hash-table (lower latency, because you simply look up the data in "constant" time , by using the key. No iteration needed.). ?

加載網站時，您希望它盡可能快和流暢。換句話說，你想低延遲 。快速查找意味著低延遲。因此，在元素數組中查找值比在哈希表中查找值要慢(延遲較高，因為您需要遍歷數組中的每個元素以查找所需的元素)(延遲較低，因為您只是看一下通過使用密鑰在“固定”時間內更新數據(無需迭代)。

Similarly, reading from memory is much faster than reading from a disk (read more here). But both have latency, and your needs will determine which type of storage you pick for which data.

同樣，從內存中讀取要比從磁盤中讀取快得多( 在此處了解更多)。但是兩者都有延遲，您的需求將決定您選擇哪種數據的存儲類型。

In that sense, latency is the inverse of speed. You want higher speeds, and you want lower latency. ?Speed (especially on network calls like via HTTP) is determined also by the distance. So, latency from London to another city, will be impacted by the distance from London.

從這個意義上講，延遲是速度的倒數。您想要更高的速度，并且想要更低的延遲。速度(尤其是通過HTTP進行的網絡呼叫)的速度也取決于距離。因此，從倫敦到另一個城市的延遲時間將受到與倫敦的距離的影響。

As you can imagine, you want to design a system to avoid pinging distant servers, but then storing things in memory may not be feasible for your system. These are the tradeoffs that make system design complex, challenging and extremely interesting!

可以想像，您希望設計一個系統來避免對遠程服務器執行ping操作，但是對于系統而言，將內容存儲在內存中可能并不可行。這些折衷使系統設計變得復雜，具有挑戰性并且非常有趣！

For example, websites that show news articles may prefer uptime and availability over loading speed, whereas online multiplayer games may require availability and super low latency. ?These requirements will determine the design and investment in infrastructure to support the system's special requirements.

例如，顯示新聞報道的網站可能更喜歡正常運行時間和可用性，而不是加載速度，而在線多人游戲可能需要可用性和超低延遲。這些要求將確定支持系統特殊要求的基礎結構的設計和投資。

通量 (Throughput)

This can be understood as the maximum capacity of a machine or system. ?It's often used in factories to calculate how much work an assembly line can do in an hour or a day, or some other unit of time measurement.

這可以理解為機器或系統的最大容量。它通常在工廠中用于計算裝配線在一小時或一天之內可以完成的工作量，或其他時間單位。

For example an assembly line can assemble 20 cars per hour, which is its throughput. In computing it would be the amount of data that can be passed around in a unit of time. ?So a 512 Mbps internet connection is a measure of throughput - 512 Mb (megabits) per second.

例如，一條裝配線每小時可以裝配20輛汽車，這就是它的吞吐量。在計算中，它是單位時間內可以傳遞的數據量。因此512 Mbps互聯網連接是吞吐量的度量-每秒512 Mb(兆位)。

Now imagine freeCodeCamp's web-server. ?If it receives 1 million requests per second, and can serve only 800,000 requests, then its throughput is 800,000 per second. You may end up measuring the throughput in terms of bits instead of requests, so it would be N bits per second.

現在想象一下freeCodeCamp的Web服務器。如果它每秒接收一百萬個請求，并且只能處理800,000個請求，則其吞吐量為每秒80萬。您可能最終以比特而不是請求來衡量吞吐量，因此它將是每秒N位。

In this example, there is a bottleneck because the server cannot handle more than N bits a second, but the requests are more than that. ?A bottleneck is therefore the constraint on a system. ?A system is only as fast as its slowest bottleneck. ?

在此示例中，存在瓶頸，因為服務器每秒處理的位數不能超過N個，但是請求的位數不止于此。因此，瓶頸是對系統的約束。一個系統的速度只有最慢的瓶頸。

If one server can handle 100 bits per second, and another can handle 120 bits per second and a third can handle only 50, then the overall system will be operating at 50bps because that is the constraint - it holds up the speed of the other servers in a given system.

如果一臺服務器每秒可以處理100位，另一臺服務器每秒可以處理120位，而第三臺服務器只能處理50位，那么整個系統將以50bps的速度運行，因為這是一個限制-它可以保持其他服務器的速度在給定的系統中。

So increasing throughput anywhere other than the bottleneck may be a waste - you may want to just increase throughput at the lowest bottleneck first. ?

因此，增加瓶頸以外的任何地方的吞吐量可能都是浪費–您可能只想首先在最低瓶頸處增加吞吐量 。

You can increase throughput by buying more hardware (horizontal scaling) or increasing the capacity and performance of your existing hardware (vertical scaling) or a few other ways.

您可以通過購買更多硬件(水平縮放)或增加現有硬件的容量和性能(垂直縮放)或其他幾種方法來提高吞吐量。

Increasing throughput may sometimes be a short term solution, and so a good systems designer will think through the best ways to scale the throughput of a given system including by splitting up requests (or any other form of "load"), and distributing them across other resources etc. The key point to remember is what throughput is, what a constraint or bottleneck is, and how it impacts a system.

有時，增加吞吐量可能是一個短期解決方案，因此，優秀的系統設計人員將考慮通過最佳方法來擴展給定系統的吞吐量，包括拆分請求(或任何其他形式的“負載”)，并將其分布在各個請求上。需要記住的關鍵點是什么是吞吐量，什么是約束或瓶頸以及它如何影響系統。

Fixing latency and throughput are not isolated, universal solutions by themselves, nor are they correlated to each other. They have impacts and considerations across the system, so it's important to understand the system as a whole, and the nature of the demands that will be placed on the system over time.

固定延遲和吞吐量不是孤立的，通用的解決方案，也不是相互關聯的。它們對整個系統都有影響和考慮因素，因此了解整個系統以及隨時間推移對系統提出的需求的性質非常重要。

第3節：系統可用性 (Section 3: System Availability)

Software engineers aim to build systems that are reliable. ?A reliable system is one that consistently satisfies a user's needs, whenever that user seeks to have that need satisfied. ?A key component of that reliability is Availability.

軟件工程師旨在構建可靠的系統。可靠的系統是一個始終滿足用戶需求的系統，只要該用戶想要滿足該需求。這種可靠性的關鍵組成部分是可用性。

It's helpful to think of availability as the resiliency of a system. ?If a system is robust enough to handle failures in the network, database, servers etc, then it can generally be considered to be a fault-tolerant system - which makes it an available system. ?

將可用性視為系統的彈性很有幫助。如果系統足夠健壯，可以處理網絡，數據庫，服務器等中的故障，則通常可以將其視為容錯系統-這使其成為可用系統。

Of course, a system is a sum of its parts in many senses, and each part needs to be highly available if availability is relevant to the end user experience of the site or app.

當然，從多種意義上講，系統是其各個部分的總和，如果可用性與站點或應用程序的最終用戶體驗有關，則每個部分都必須具有高可用性。

量化可用性 (Quantifying Availability)

To quantify the availability of a system, we calculate the percentage of time that the system's primary functionality and operations are available (the uptime) in a given window of time.

為了量化系統的可用性，我們在給定的時間范圍內計算系統的主要功能和操作可用的時間百分比(正常運行時間)。

The most business-critical systems would need to have a near-perfect availability. Systems that support highly variable demands and loads with sharp peaks and troughs may be able to get away with slightly lower availability during off-peak times.

最關鍵的業務系統將需要具有接近完美的可用性。在高峰時段，支持高變化需求和尖峰和低谷負載的系統可能會以較低的可用性擺脫困境。

It all depends on the use and nature of the system. But in general, even things that have low, but consistent demands or an implied guarantee that the system is "on-demand" would need to have high availability.

這完全取決于系統的用途和性質。但是總的來說，即使是那些需求較低但始終如一或隱含保證系統“按需”的事物，也需要具有高可用性。

Think of a site where you backup your pictures. ?You don't always need to access and retrieve data from this - it's mainly for you to store things in. ?You would still expect it to always be available any time you login to download even just a single picture. ?

想想一個備份圖片的網站。您并非總是需要從中訪問和檢索數據-主要是用于存儲內容。您仍然希望它在每次登錄下載任何圖片時始終可用。

A different kind of availability can be understood in the context of the massive e-commerce shopping days like Black Friday or Cyber Monday sales. ?On these particular days demand will skyrocket and millions will try to access the deals simultaneously. ?That would require an extremely reliable and high-availability system design to support those loads.

在黑色星期五或網絡星期一銷售這樣的大型電子商務購物日中，可以理解另一種可用性。在這些特定日子里，需求將激增，數以百萬計的人將嘗試同時訪問交易。這將需要極其可靠和高可用性的系統設計來支持這些負載。

A commercial reason for high availability is simply that any downtime on the site will result in the site losing money. ?Also, it could be really bad for reputation, for example, where the service is a service used by other businesses to offer services. ?If AWS S3 goes down, a lot of companies will suffer, including Netflix, and that is not good.

高可用性的商業原因僅僅是因為該站點上的任何停機都將導致該站點虧本。此外，這可能對聲譽確實不利，例如，該服務是其他企業用來提供服務的服務。如果AWS S3出現故障，包括Netflix在內的許多公司將遭受損失，這不是很好 。

So uptimes are extremely important for success. ?It is worth remembering that commercial availability numbers ?are calculated based on ?annual availability, so a downtime of 0.1% (i.e. availability of 99.9%) is 8.77 hours a year!

因此，正常運行時間對于成功至關重要。值得記住的是，商業可用性數字是根據年度可用性計算得出的，因此0.1％(即99.9％的可用性)的停機時間是每年8.77個小時！

Hence, uptimes are extremely high sounding. ?It is common to see things like 99.99% uptime (52.6 minutes of downtime per year). ? Which is why it is now common to refer to uptimes in terms of "nines" - the number of nines in the uptime assurance. ?

因此，正常運行時間非常高。通常會看到99.99％的正常運行時間(每年52.6分鐘的停機時間)。這就是為什么現在通常用“ nines”(正常運行時間保證中的九位數)來指稱正常運行時間。

In today's world that is unacceptable for large-scale or mission critical services. ?Which is why these days "five nines" is considered the ideal availability standard because that translates to a little over 5 minutes of downtime per year.

在當今世界上，大規模或關鍵任務服務是不可接受的。這就是為什么現在“五個九”被認為是理想的可用性標準的原因，因為這相當于每年停機時間超過5分鐘。

服務水平協議 (SLAs)

In order to make online services competitive and meet the market's expectations, online service providers typically offer Service Level Agreements/Assurances. ?These are a set of guaranteed service level metrics. ?99.999% uptime is one such metric and is often offered as part of premium subscriptions.

為了使在線服務具有競爭力并滿足市場的期望，在線服務提供商通常會提供服務水平協議/保證。這些是一組保證的服務級別指標。 99.999％的正常運行時間就是這樣一種指標，通常作為高級訂閱的一部分提供。

In the case of database and cloud service providers this can be offered even on the trial or free tiers if a customer's core use for that product justifies the expectation of such a metric.

對于數據庫和云服務提供商，如果客戶對該產品的核心使用證明了該指標的合理性，則即使在試用或免費套餐上也可以提供此服務。

In many cases failing to meet the SLA will give the customer a right to credits or some other form of compensation for the provider's failure to meet that assurance. ?Here, by way of example, is Google's SLA for the Maps API.

在許多情況下，如果未能滿足SLA，則客戶將獲得信用證或其他形式的補償，以彌補提供商未達到該保證的權利。舉例來說，這里是Google針對Maps API的SLA。

SLAs are therefore a critical part of the overall commercial and technical consideration when designing a system. It is especially important to consider whether availability is in fact a key requirement for a part of a system, and which parts require high availability.

因此，在設計系統時，SLA是整個商業和技術考慮因素的關鍵部分。特別重要的是考慮可用性是否實際上是系統某個部分的關鍵要求，以及哪些部分需要高可用性。

設計HA (Designing HA)

When designing a high availability (HA) system, then, you need to reduce or eliminate "single points of failure". ?A single point of failure is an element in the system that is the sole element that can produce that undesirable loss of availability.

因此，在設計高可用性(HA)系統時，您需要減少或消除“單點故障”。單點故障是系統中的一個元素，它是唯一可能導致不希望的可用性損失的元素。

You eliminate single points of failure by designing 'redundancy' into the system. Redundancy is basically making 1 or more alternatives (i.e. backups) to the element that is critical for high availability. ?

您可以通過在系統中設計“冗余”來消除單點故障。冗余基本上是對對高可用性至關重要的元素進行1個或多個選擇(即備份)。

So if your app needs users to be authenticated to use it, and there is only one authentication service and back end, and that fails, then, because that is the single point of failure, your system is no longer usable. ?By having two or more services that can handle authentication, you have added redundancy and eliminated (or reduced) single points of failure.

因此，如果您的應用需要用戶進行身份驗證才能使用它，并且只有一個身份驗證服務和后端，并且該服務失敗，則因為這是單點故障，所以您的系統不再可用。通過擁有兩個或多個可以處理身份驗證的服務，您增加了冗余并消除了(或減少了)單點故障。

Therefore, you need to understand and de-compose your system into all its parts. Map out which ones are likely to cause single points of failure, which ones are not tolerant of such failure, and which parts can tolerate them. Because engineering HA requires tradeoffs and some of these tradeoffs may be expensive in terms of time, money and resources.

因此，您需要了解系統并將其分解為所有部分。找出哪些可能導致單點故障，哪些不能容忍這種故障，哪些部分可以容忍它們。由于工程HA需要權衡，并且其中一些權衡在時間，金錢和資源方面可能是昂貴的。

第4節：緩存 (Section 4: Caching)

Caching! This is a very fundamental and easy-to-understand technique to speed up performance in a system. ?Thus caching helps to reduce "latency" in a system.

正在緩存！這是提高系統性能的非常基礎且易于理解的技術。因此，緩存有助于減少系統中的“等待時間” 。

In our daily lives, we use caching as a matter of common-sense (most of the time...). If we live next door to a supermarket, we still want to buy and store some basics in our fridge and our food cupboard. ?This is caching. ?We could always step out, go next door, and buy these things every time we want food – but if its in the pantry or fridge, we reduce the time it takes to make our food. ?That's caching.

在我們的日常生活中，我們將緩存作為常識(在大多數情況下...)。如果我們住在超市的隔壁，我們仍然想在冰箱和食物柜中購買和存儲一些基本物品。這是緩存。每當我們想要食物時，我們總是可以出門，去隔壁并購買這些東西-但是，如果它在食品室或冰箱中，我們會減少制作食物的時間。那是緩存。

常見的緩存方案 (Common Scenarios for Caching)

Similarly, in software terms, if we end up relying on certain pieces of data often, we may want to cache that data so that our app performs faster.

同樣，就軟件而言，如果我們最終經常依賴某些數據，則可能需要緩存該數據，以便我們的應用程序執行得更快。

This is often true when it's faster to retrieve data from Memory rather than disk because of the latency in making network requests. In fact many websites are cached (especially if content doesn't change frequently) in CDNs so that it can be served to the end user much faster, and it reduces load on the backend servers. ?

當由于發出網絡請求的延遲而從內存而不是從磁盤檢索數據更快時，通常會如此。實際上，許多網站都在CDN中緩存(尤其是內容不經常更改的情況)，以便可以更快地將其提供給最終用戶，并減少了后端服務器的負載。

Another context in which caching helps could be where your backend has to do some computationally intensive and time consuming work. Caching previous results that converts your lookup time from a linear O(N) time to constant O(1) time could be very advantageous.

緩存可以幫助您解決的另一個問題是后端必須執行一些計算量大且耗時的工作。緩存將您的查找時間從線性O(N)時間轉換為恒定O(1)時間的先前結果可能會非常有優勢。

Likewise, if your server has to make multiple network requests and API calls in order to compose the data that gets sent back to the requester, then caching data could reduce the number of network calls, and thus the latency.

同樣，如果您的服務器必須進行多個網絡請求和API調用才能組成發送回請求者的數據，則緩存數據可以減少網絡調用的次數，從而減少延遲。

If your system has a client (front end), and a server and databases (backend) then caching can be inserted on the client (e.g. browser storage), between the client and the server (e.g. CDNs), or on the server itself. This would reduce over-the-network calls to the database. ?

如果您的系統具有客戶端(前端)，服務器和數據庫(后端)，則可以在客戶端(例如瀏覽器存儲)，客戶端和服務器之間(例如CDN)或服務器本身上插入緩存。這將減少對數據庫的網絡調用。

So caching can occur at multiple points or levels in the system, including at the hardware (CPU) level.

因此，緩存可以在系統中的多個點或級別上發生，包括在硬件(CPU)級別上。

處理陳舊數據 (Handling Stale Data)

You may have noticed that the above examples are implicitly handy for "read" operations. ?Write operations are not that different, in main principles, with the following added considerations:

您可能已經注意到，上面的示例對于“讀取”操作是隱式方便的。寫入操作在主要原則上并沒有什么不同，但還要增加以下注意事項：

write operations require keeping the cache and your database in sync
寫操作需要保持緩存和數據庫同步
this may increase complexity because there are more operations to perform, and new considerations around handling un-synced or "stale" data need to be carefully analyzed
這可能會增加復雜性，因為要執行更多的操作，并且需要仔細分析有關處理未同步或“過時”數據的新注意事項
new design principles may need to be implemented to handle that syncing - should it be done synchronously, or asynchronously? If async, then at what intervals? Where does data get served from in the mean time? How often does the cache need to be refreshed, etc...
可能需要實施新的設計原則來處理該同步-應該同步還是異步完成？如果異步，那么以什么間隔？在此期間從何處獲取數據？緩存需要多久刷新一次，等等。
data "eviction" or turnover and refreshes of data, to keep cached data fresh and up-to-date. These include techniques like LIFO, FIFO, LRU and LFU.
數據“逐出”或更新和刷新數據，以使緩存的數據保持最新和最新狀態。這些包括LIFO ， FIFO ， LRU和LFU之類的技術。

So let's end with some high-level, and non-binding conclusions. Generally, caching works best when used to store static or infrequently changing data, and when the sources of change are likely to be single operations rather than user-generated operations. ?

因此，讓我們以一些概括性的，非約束性的結論作為結尾。通常，當用于存儲靜態或不經常更改的數據時，并且當更改的源可能是單個操作而不是用戶生成的操作時，緩存效果最佳。

Where consistency and freshness in data is critical, caching may not be an optimal solution, unless there is another element in the system that efficiently refreshes the caches are intervals that do not adversely impact the purpose and user experience of the application.

在數據的一致性和新鮮度至關重要的情況下，緩存可能不是最佳的解決方案，除非系統中有另一個有效刷新緩存的元素是不會對應用程序的目的和用戶體驗產生不利影響的間隔。

第5節：代理 (Section 5: Proxies)

Proxy. What? ?Many of us have heard of proxy servers. ?We may have seen configuration options on some of our PC or Mac software that talk about adding and configuring proxy servers, or accessing "via a proxy".

代理。什么？我們許多人都聽說過代理服務器。我們可能已經在某些PC或Mac軟件上看到了配置選項，這些選項討論添加和配置代理服務器或“通過代理”訪問。

So let's understand that relatively simple, widely used and important piece of tech. This is a word that exists in the English language completely independent of computer science, so let's start with that definition.

因此，讓我們了解一下相對簡單，廣泛使用且重要的技術。這是一個完全獨立于計算機科學的英語單詞，因此讓我們從該定義開始。

Now you can eject most of that out of your mind, and hold on to one key word: "substitute". ?

現在，您可以將大部分內容從頭腦中彈出，然后按住一個關鍵字：“替換”。

In computing, a proxy is typically a server, and it is a server that acts as a middleman between a client and another server. It literally is a bit of code that sits between client and server. That's the crux of proxies.

在計算中，代理通常是服務器，并且它是充當客戶端和另一臺服務器之間的中間人的服務器。從字面上看，這是位于客戶端和服務器之間的一些代碼。這就是代理的癥結所在。

In case you need a refresher, or aren't sure of the definitions of client and server, a "client" is a process (code) or machine that requests data from another process or machine (the "server"). ?The browser is a client when it requests data from a backend server. ?

如果您需要復習，或者不確定客戶端和服務器的定義，則“客戶端”是從另一個進程或計算機(“服務器”)請求數據的進程(代碼)或計算機。當瀏覽器從后端服務器請求數據時，它是客戶端。

The server serves the client, but can also be a client - when it retrieves data from a database. Then the database is the server, the server is the client (of the database) and also a server for the front-end client (browser).

服務器為客戶端提供服務，但也可以是客戶端-從數據庫檢索數據時。然后，數據庫是服務器，服務器是(數據庫的)客戶端，也是前端客戶端(瀏覽器)的服務器。

As you can see from the above, the client-server relationship is bi-directional. ?So one things can be both the client and server. ?If there was a middleman server that received requests, then sent them to another service, then forwards the response it got from that other service back to the originator client, that would be a proxy server.

從上面可以看到，客戶端-服務器關系是雙向的。因此，客戶端和服務器都是一回事。如果有一個中間人服務器接收到請求，然后將它們發送到另一個服務，然后將它從另一個服務獲得的響應轉發回原始客戶端，即代理服務器。

Going forward we will refer to clients as clients, servers as servers and proxies as the thing between them.

展望未來，我們將客戶端稱為客戶端，將服務器稱為服務器，將代理稱為它們之間的事物。

So when a client sends a request to a server via the proxy, the proxy may sometimes mask the identity of the client - to the server, the IP address that comes through in the request may be the proxy and not the originating client.

因此，當客戶端通過代理向服務器發送請求時，代理有時會掩蓋客戶端的身份-到服務器時，請求中通過的IP地址可能是代理而不是原始客戶端。

For those of you who access sites or download things that otherwise are restricted (from the torrent network for example, or sites banned in your country), you may recognize this pattern - it's the principle on which VPNs are built.

對于那些訪問網站或下載其他限制內容的人(例如從torrent網絡或您所在國家/地區禁止的網站)，您可能會意識到這種模式-這是建立VPN的原理。

Before we move a bit deeper, I want to call something out - when generally used, the term proxy refers to a "forward" proxy. ?A forward proxy is one where the proxy acts on behalf of (substitute for) the client in the interaction between client and server.

在深入探討之前，我想先介紹一下-通常使用的術語“代理”是指“轉發”代理。轉發代理是指代理在客戶端與服務器之間的交互中代表客戶端(代替客戶端)進行操作的代理。

This is distinguished from a reverse proxy - where the proxy acts on behalf of a server. ?On a diagram it would look the same - the proxy sits between the client and the server, and the data flows are the same client <-> proxy <-> server. ?

這與反向代理不同-反向代理代表服務器。在圖中，它看起來是一樣的-代理位于客戶端和服務器之間，并且數據流是相同的客戶端<->代理<->服務器。

The key difference is that a reverse proxy is designed substitute for the server. ?Often clients won't even know that the network request got routed through a proxy and the proxy passed it on to the intended server (and did the same thing with the server's response).

關鍵區別在于反向代理被設計為替代服務器。通常，客戶甚至都不知道網絡請求是通過代理路由的，并且代理將其傳遞給了預期的服務器(并且對服務器的響應做了同樣的事情)。

So, in a forward proxy, the server won't know that the client's request and its response are traveling through a proxy, and in a reverse proxy the client won't know that the request and response are routed through a proxy.

因此，在正向代理中，服務器將不知道客戶端的請求及其響應正在通過代理，而在反向代理中，客戶端將不知道請求和響應是通過代理進行路由的。

Proxies feel kinda sneaky :)

代理感到有點偷偷摸摸:)

But in systems design, especially for complex systems, proxies are useful and reverse proxies are particularly useful. Your reverse proxy can be delegated a lot of tasks that you don't want your main server handling - it can be a gatekeeper, a screener, a load-balancer and an all around assistant.

但是在系統設計中，特別是對于復雜系統，代理是有用的，反向代理是特別有用的。您的反向代理可以委派許多您不希望主服務器處理的任務-它可以是網守，篩選器，負載平衡器和全方位助手。

So proxies can be useful but you may not be sure why. ?Again, if you've read my other stuff you'd know that I firmly believe that you can understand things properly only when you know why they exist - knowing what they do is not enough. ?

因此代理可能有用，但您可能不確定原因。同樣，如果你讀過我的其他的東西，你知道，我堅信你能正確地只了解事情的時候，你知道他們為什么存在-知道他們做的是不夠的。

We've talked about VPNs (for forward proxies) and load-balancing (for reverse proxies), but there are more examples here - I particularly recommend Clara Clarkson's high level summary.

我們已經談到的VPN(用于轉發代理)和負載平衡(反向代理服務器)，但也有更多的例子在這里 -我特別推薦克拉拉Clarkson的高度概括。

第6節：負載平衡 (Section 6: Load Balancing)

If you think about the two words, load and balance, you will start to get an intuition as to what this does in the world of computing. ?When a server simultaneously receives a lot of requests, it can slow down (throughput reduces, latency rises). ?After a point it may even fail (no availability). ?

如果您考慮一下負載和平衡這兩個詞，您將開始對它在計算領域的作用有一個直覺。當服務器同時接收到大量請求時，它可能會減慢速度(吞吐量降低，延遲增加)。在一個點之后，它甚至可能失敗(無可用性)。

You can give the server more muscle power (vertical scaling) or you can add more servers (horizontal scaling). ?But now you got to work out how the income requests get distributed to the various servers - which requests get routed to which servers and how to ensure they don't get overloaded too? In other words, how do you balance and allocate the request load?

您可以賦予服務器更多的力量(垂直縮放)，也可以添加更多服務器(水平縮放)。但是現在您必須弄清楚收入請求如何分配到各個服務器-哪些請求被路由到哪些服務器以及如何確保它們也不會過載？換句話說，如何平衡和分配請求負載？

Enter load balancers. Since this article is an introduction to principles and concepts, they are, of necessity, very simplified explanations. A load balancer's job is to sit between the client and server (but there are other places it can be inserted) and work out how to distribute incoming request loads across multiple servers, so that the end user (client's) experience is consistently fast, smooth and reliable.

輸入負載均衡器。由于本文是對原理和概念的介紹，因此它們在必要時必須經過非常簡化的解釋。負載平衡器的工作是坐在客戶端和服務器之間(但可以在其他位置插入它)，并研究如何在多個服務器之間分配傳入請求負載，以便最終用戶(客戶端)的體驗始終快速，流暢和可靠。

So load balancers are like traffic managers who direct traffic. ?And they do this to maintain availability and throughput.

因此，負載均衡器就像是引導流量的流量管理器。他們這樣做是為了保持可用性和吞吐量。

When understanding where a load balancer is inserted in the system's architecture, you can see that load balancers can be thought of as reverse proxies. ?But a load balancer can be inserted in other places too - between other exchanges - for example, between your server and your database.

了解負載均衡器在系統體系結構中的插入位置后，您會發現負載均衡器可以被視為反向代理。但是負載平衡器也可以插入其他位置，例如在其他交換機之間，例如在服務器和數據庫之間。

平衡法-服務器選擇策略 (The Balancing Act - Server Selection Strategies)

So how does the load balancer decide how to route and allocate request traffic? To start with, every time you add a server, you need to let your load balancer know that there is one more candidate for it to route traffic to. ?

那么，負載均衡器如何決定如何路由和分配請求流量？首先，每次添加服務器時，都需要讓負載均衡器知道它還有一個候選者可以將流量路由到該負載均衡器。

If you remove a server, the load balancer needs to know that too. ?The configuration ensures that the load balancer knows how many servers it has in its go-to list and which ones are available. ?It is even possible for the load balancer to be kept informed on each server's load levels, status, availability, current task and so on.

如果卸下服務器，則負載平衡器也需要知道這一點。該配置可確保負載均衡器知道其轉到列表中有多少臺服務器以及哪些服務器可用。甚至可以使負載平衡器隨時了解每臺服務器的負載級別，狀態，可用性，當前任務等。

Once the load balancer is configured to know what servers it can redirect to, we need to work out the best routing strategy to ensure there is proper distribution amongst the available servers. ?

將負載均衡器配置為知道可以重定向到哪些服務器后，我們需要制定出最佳路由策略，以確保在可用服務器之間進行適當的分配。

A naive approach to this is for the load balancer to just randomly pick a server and direct each incoming request that way. ?But as you can imagine, randomness can cause problems and "unbalanced" allocations where some servers get more loaded than others, and that could affect performance of the overall system negatively.

一個簡單的方法是讓負載均衡器隨機選擇一個服務器，然后以這種方式定向每個傳入的請求。但是正如您可以想象的那樣，隨機性可能會導致問題和“不平衡”分配，其中某些服務器的負載要比其他服務器更多，這可能會對整個系統的性能產生負面影響。

循環賽和加權循環賽 (Round Robin and Weighted Round Robin)

Another method that can be intuitively understood is called "round robin". This is the way many humans process lists that loop. ?You start at the first item in the list, move down in sequence, and when you're done with the last item you loop back up to the top and start working down the list again.

可以直觀理解的另一種方法稱為“循環”。這是許多人處理列表循環的方式。您從列表中的第一項開始，依次向下移動，當最后一項完成后，您將循環回到頂部，然后再次開始處理列表。

The load balancer can do this too, by just looping through available servers in a fixed sequence. ?This way the load is pretty evenly distributed across your servers in a simple-to-understand and predictable pattern. ?

負載均衡器也可以做到這一點，只需按固定順序遍歷可用服務器即可。這樣，負載就以一種易于理解和可預測的模式在服務器上平均分配。

You can get a little more "fancy" with the round robin by "weighting" some services over others. ?In the normal, standard round robin, each server is given equal weight (let's say all are given a weighting of 1). ?But when you differently weight servers, then you can have some servers with a lower weighting (say 0.5, if they're less powerful), ?and others can be higher like 0.7 or 0.9 or even 1.

通過將某些服務“加權”于其他服務，可以使輪詢變得更“花哨”。在正常的標準輪詢中，每個服務器的權重相等(假設所有服務器的權重均為1)。但是，當您對服務器加權時，可以使某些服務器的權重較低(例如，如果它們的功能較弱，則為0.5)，而其他服務器的權重可能更高，例如0.7或0.9甚至是1。

Then the total traffic will be split up in proportion to those weights and allocated accordingly to the servers that have power proportionate to the volume of requests.

然后，總流量將按這些權重成比例分配，并相應地分配給功率與請求量成正比的服務器。

基于負載的服務器選擇 (Load-based server selection)

More sophisticated load balancers can work out the current capacity, performance, and loads of the servers in their go-to list and allocate dynamically according to current loads and calculations as to which will have the highest throughput, lowest latency etc. It would do this by monitoring the performance of each server and deciding which ones can and cannot handle the new requests. ?

更復雜的負載均衡器可以在其go-to列表中計算出服務器的當前容量，性能和負載，并根據當前負載和計算動態分配，從而得出吞吐量最高，延遲最低等。 by monitoring the performance of each server and deciding which ones can and cannot handle the new requests.

IP Hashing based selection (IP Hashing based selection)

You can configure your load balancer to hash the IP address of incoming requests, and use the hash value to determine which server to direct the request too. ?If I had 5 servers available, then the hash function would be designed to return one of five hash values, so one of the servers definitely gets nominated to process the request.

You can configure your load balancer to hash the IP address of incoming requests, and use the hash value to determine which server to direct the request too. If I had 5 servers available, then the hash function would be designed to return one of five hash values, so one of the servers definitely gets nominated to process the request.

IP hash based routing can be very useful where you want requests from a certain country or region to get data from a server that is best suited to address the needs from within that region, or where your servers cache requests so that they can be processed fast. ?

In the latter scenario, you want to ensure that the request goes to a server that has previously cached the same request, as this will improve speed and performance in processing and responding to that request.

If your servers each maintain independent caches and your load balancer does not consistently send identical requests to the same server, you will end up with servers re-doing work that has already been done in as previous request to another server, and you lose the optimization that goes with caching data.

Path or Service based selection (Path or Service based selection)

You can also get the load balancer to route requests based on their "path" or function or service that is being provided. ?For example if you're buying flowers from an online florist, requests to load the "Bouquets on Special" may be sent to one server and credit card payments may be sent to another server.

You can also get the load balancer to route requests based on their "path" or function or service that is being provided. For example if you're buying flowers from an online florist, requests to load the "Bouquets on Special" may be sent to one server and credit card payments may be sent to another server.

If only one in twenty visitors actually bought flowers, then you could have a smaller server processing the payments and a bigger one handling all the browsing traffic.

Mixed Bag (Mixed Bag)

And as with all things, you can get to higher and more detailed levels of complexity. You can have multiple load balancers that each have different server selection strategies! ?And if yours is a very large and highly trafficked system, then you may need load balancers for load balancers...

And as with all things, you can get to higher and more detailed levels of complexity. You can have multiple load balancers that each have different server selection strategies! And if yours is a very large and highly trafficked system, then you may need load balancers for load balancers...

Ultimately, you add pieces to the system until your performance is tuned to your needs (your needs may look flat, or slow upwards mildly over time, or be prone to spikes!).

Section 7: Consistent Hashing (Section 7: Consistent Hashing)

One of the slightly more tricky concepts to understand is hashing in the context of load balancing. So it gets its own section.

In order to understand this, please first understand how hashing works at a conceptual level. The TL;DR is that hashing converts an input into a fixed-size value, often an integer value (the hash). ?

In order to understand this, please first understand how hashing works at a conceptual level . The TL;DR is that hashing converts an input into a fixed-size value, often an integer value (the hash).

One of the key principles for a good hashing algorithm or function is that the function must be deterministic, which is a fancy way for saying that identical inputs will generate identical outputs when passed into the function. So, deterministic means - if I pass in the string "Code" (case sensitive) and the function generates a hash of 11002, then every time I pass in "Code" it must generate "11002" as an integer. And if I pass in "code" it will generate a different number (consistently).

One of the key principles for a good hashing algorithm or function is that the function must be deterministic , which is a fancy way for saying that identical inputs will generate identical outputs when passed into the function. So, deterministic means - if I pass in the string "Code" (case sensitive) and the function generates a hash of 11002, then every time I pass in "Code" it must generate "11002" as an integer. And if I pass in "code" it will generate a different number (consistently).

Sometimes the hashing function can generate the same hash for more than one input - this is not the end of the world and there are ways to deal with it. ?In fact it becomes more likely the more the range of unique inputs are. ?But when more than one input deterministically generates the same output, it's called a "collision".

Sometimes the hashing function can generate the same hash for more than one input - this is not the end of the world and there are ways to deal with it. In fact it becomes more likely the more the range of unique inputs are. But when more than one input deterministically generates the same output, it's called a "collision".

With this in firmly in mind, let's apply it to routing and directed requests to servers. Let's say you have 5 servers to allocate loads across. ?An easy to understand method would be to hash incoming requests (maybe by IP address, or some client detail), and then generate hashes for each request. ?Then you apply the modulo operator to that hash, where the right operand is the number of servers. ?

With this in firmly in mind, let's apply it to routing and directed requests to servers. Let's say you have 5 servers to allocate loads across. An easy to understand method would be to hash incoming requests (maybe by IP address, or some client detail), and then generate hashes for each request. Then you apply the modulo operator to that hash, where the right operand is the number of servers.

For example, this is what your load balancers' pseudo code could look like:

request#1 => hashes to 34
request#2 => hashes to 23
request#3 => hashes to 30
request#4 => hashes to 14// You have 5 servers => [Server A, Server B ,Server C ,Server D ,Server E]// so modulo 5 for each request...request#1 => hashes to 34 => 34 % 5 = 4 => send this request to servers[4] => Server Erequest#2 => hashes to 23 => 23 % 5 = 3 => send this request to servers[3] => Server Drequest#3 => hashes to 30 => 30 % 5 = 0 => send this request to  servers[0] => Server Arequest#4 => hashes to 14 => 14 % 5 = 4 => send this request to servers[4] => Server E

As you can see, the hashing function generates a spread of possible values, and when the modulo operator is applied it brings out a smaller range of numbers that map to the server number.

You will definitely get different requests that map to the same server, and that's fine, as long as there is "uniformity" in the overall allocation to all the servers.

You will definitely get different requests that map to the same server, and that's fine, as long as there is " uniformity " in the overall allocation to all the servers.

Adding Servers, and Handling Failing Servers (Adding Servers, and Handling Failing Servers)

So - what happens if one of the servers that we are sending traffic to dies? The hashing function (refer to the pseudo code snippet above) still thinks there are 5 servers, and the mod operator generates a range from 0-4. ?But we only have 4 servers now that one has failed, and we are still sending it traffic. ?Oops.

So - what happens if one of the servers that we are sending traffic to dies? The hashing function (refer to the pseudo code snippet above) still thinks there are 5 servers, and the mod operator generates a range from 0-4. But we only have 4 servers now that one has failed, and we are still sending it traffic. 哎呀。

Inversely, we could add a sixth server but that would never get any traffic because our mod operator is 5, and it will never yield a number that would include the newly added 6th server. Double oops.

// Let's add a 6th server
servers => [Server A, Server B ,Server C ,Server D ,Server E, Server F]// let's change the modulo operand to 6
request#1 => hashes to 34 => 34 % 6 = 4 => send this request to servers[4] => Server Erequest#2 => hashes to 23 => 23 % 6 = 5 => send this request to servers[5] => Server Frequest#3 => hashes to 30 => 30 % 6 = 0 => send this request to  servers[0] => Server Arequest#4 => hashes to 14 => 14 % 6 = 2 => send this request to servers[2] => Server C

We note that the server number after applying the mod changes (though, in this example, not for request#1 and request#3 - but that is just because in this specific case the numbers worked out that way).

In effect, the result is that half the requests (could be more in other examples!) are now being routed to new servers altogether, and we lose the benefits of previously cached data on the servers. ?

In effect, the result is that half the requests (could be more in other examples!) are now being routed to new servers altogether, and we lose the benefits of previously cached data on the servers.

For example, request#4 used to go to Server E, but now goes to Server C. ?All the cached data relating to request#4 sitting on Server E is of no use since the request is now going to Server C. ?You can calculate a similar problem for where one of your servers dies, but the mod function keeps sending it requests.

For example, request#4 used to go to Server E, but now goes to Server C. All the cached data relating to request#4 sitting on Server E is of no use since the request is now going to Server C. You can calculate a similar problem for where one of your servers dies, but the mod function keeps sending it requests.

It sounds minor in this tiny system. ?But on a very large scale system this is a poor outcome. #SystemDesignFail.

It sounds minor in this tiny system. But on a very large scale system this is a poor outcome. #SystemDesignFail.

So clearly, a simple hashing-to-allocate system does not scale or handle failures well.

A popular solution - consistent hashing (A popular solution - consistent hashing)

Unfortunately this is the part where I feel word descriptions will not be enough. Consistent hashing is best understood visually. ?But the purpose of this post so far is to give you an intuition around the problem, what it is, why it arises, and what the shortcomings in a basic solution might be. ?Keep that firmly in mind.

Unfortunately this is the part where I feel word descriptions will not be enough. Consistent hashing is best understood visually. But the purpose of this post so far is to give you an intuition around the problem, what it is, why it arises, and what the shortcomings in a basic solution might be. Keep that firmly in mind.

The key problem with naive hashing, as we discussed, is that when (A) a server fails, traffic still gets routed to it, and (B) you add a new server, the allocations can get substantially changed, thus losing the benefits of previous caches.

There are two very important things to keep in mind when digging into consistent hashing:

Consistent hashing does not eliminate the problems, especially B. But it does reduce the problems a lot. At first you might wonder what the big deal is in consistent hashing, as the underlying downside still exists - yes, but to a much smaller extent, and that itself is a valuable improvement in very large scale systems.
Consistent hashing does not eliminate the problems , especially B. But it does reduce the problems a lot. At first you might wonder what the big deal is in consistent hashing, as the underlying downside still exists - yes, but to a much smaller extent, and that itself is a valuable improvement in very large scale systems.
Consistent hashing applies a hash function to incoming requests and the servers. The resulting outputs therefore fall in a set range (continuum) of values. ?This detail is very important.
Consistent hashing applies a hash function to incoming requests and the servers . The resulting outputs therefore fall in a set range (continuum) of values. This detail is very important.

Please keep these in mind as you watch the below recommended video that explains consistent hashing, as otherwise its benefits may not be obvious.

I strongly recommend this video as it embeds these principles without burdening you with too much detail.

If you're having a little trouble really understanding why this strategy is important in load balancing, I suggest you take a break, then return to the load balancing section and then re-read this again. ?It's not uncommon for all this to feel very abstract unless you've directly encountered the problem in your work!

If you're having a little trouble really understanding why this strategy is important in load balancing, I suggest you take a break, then return to the load balancing section and then re-read this again. It's not uncommon for all this to feel very abstract unless you've directly encountered the problem in your work!

Section 8: Databases (Section 8: Databases)

We briefly considered that there are different types of storage solutions (databases) designed to suit a number of different use-cases, and some are more specialized for certain tasks than others. ?At a very high level though, databases can be categorized into two types: Relational and Non-Relational. ?

We briefly considered that there are different types of storage solutions (databases) designed to suit a number of different use-cases, and some are more specialized for certain tasks than others. At a very high level though, databases can be categorized into two types: Relational and Non-Relational.

Relational Databases (Relational Databases)

A relational database is one that has strictly enforced relationships between things ?stored in the database. These relationships are typically made possible by requiring the database to represented each such thing (called the "entity") as a structured table - with zero or more rows ("records", "entries") and and one or more columns ("attributes, "fields").

A relational database is one that has strictly enforced relationships between things stored in the database. These relationships are typically made possible by requiring the database to represented each such thing (called the "entity") as a structured table - with zero or more rows ("records", "entries") and and one or more columns ("attributes, "fields").

By forcing such a structure on an entity, we can ensure that each item/entry/record has the right data to go with it. ?It makes for better consistency and the ability to make tight relationships between the entities. ?

By forcing such a structure on an entity, we can ensure that each item/entry/record has the right data to go with it. It makes for better consistency and the ability to make tight relationships between the entities.

You can see this structure in the table recording "Baby" (entity) data below. ?Each record ("entry) in the table has 4 fields, which represent data relating to that baby. This is a classic relational database structure (and a formalized entity structure is called a schema).

You can see this structure in the table recording "Baby" (entity) data below. Each record ("entry) in the table has 4 fields, which represent data relating to that baby. This is a classic relational database structure (and a formalized entity structure is called a schema ).

So the key feature to understand about relational databases is that they are highly structured, and impose structure on all the entities. ?This structure in enforced by ensuring that data added to the table conforms to that structure. ?Adding a height field to the table when its schema doesn't allow for it will not be permitted.

So the key feature to understand about relational databases is that they are highly structured, and impose structure on all the entities. This structure in enforced by ensuring that data added to the table conforms to that structure. Adding a height field to the table when its schema doesn't allow for it will not be permitted.

Most relational databases support a database querying language called SQL - Structured Query Language. This is a language specifically designed to interact with the contents of a structured (relational) database. The two concepts are quite tightly coupled, so much so that people often referred to a relational database as a "SQL database" (and sometimes pronounced as "sequel" database). ?

Most relational databases support a database querying language called SQL - Structured Query Language . This is a language specifically designed to interact with the contents of a structured (relational) database. The two concepts are quite tightly coupled, so much so that people often referred to a relational database as a "SQL database" (and sometimes pronounced as "sequel" database).

In general, it is considered that SQL (relational) databases support more complex queries (combining different fields and filters and conditions) than non-relational databases. The database itself handles these queries and sends back matching results. ?

Many people who are SQL database fans argue that without that function, you would have to fetch all the data and then have the server or the client load that data "in memory" and apply the filtering conditions - which is OK for small sets of data but for a large, complex dataset, with millions of records and rows, that would badly affect performance. However, this is not always the case, as we will see when we learn about NoSQL databases.

Many people who are SQL database fans argue that without that function, you would have to fetch all the data and then have the server or the client load that data " in memory " and apply the filtering conditions - which is OK for small sets of data but for a large, complex dataset, with millions of records and rows, that would badly affect performance. However, this is not always the case, as we will see when we learn about NoSQL databases.

A common and much-loved example of a relational database is the PostgreSQL (often called "Postgres") database. ?

A common and much-loved example of a relational database is the PostgreSQL (often called "Postgres") database.

ACID (ACID)

ACID transactions are a set of features that describe the transactions that a good relational database will support. ACID = "Atomic, Consistent, Isolation, Durable". A transaction is an interaction with a database, typically read or write operations.

ACID transactions are a set of features that describe the transactions that a good relational database will support. ACID = "Atomic, Consistent, Isolation, Durable" . A transaction is an interaction with a database, typically read or write operations.

Atomicity requires that when a single transaction comprises of more than one operation, then the database must guarantee that if one operation fails the entire transaction (all operations) also fail. ?It's "all or nothing". That way if the transaction succeeds, then on completion you know that all the sub-operations completed successfully, and if an operation fails, then you know that all the operations that went with it failed. ?

Atomicity requires that when a single transaction comprises of more than one operation, then the database must guarantee that if one operation fails the entire transaction (all operations) also fail. It's "all or nothing". That way if the transaction succeeds, then on completion you know that all the sub-operations completed successfully, and if an operation fails, then you know that all the operations that went with it failed.

For example if a single transaction involved reading from two tables and writing to three, then if any one of those individual operations fails the entire transaction fails. This means that none of those individual operations should complete. You would not want even 1 out of the 3 write transactions to work - that would "dirty" the data in your databases!

Consistency requires that each transaction in a database is valid according to the database's defined rules, and when the database changes state (some information has changed), such change is valid and does not corrupt the data. Each transaction moves the database from one valid state to another valid state. Consistency can be thought of as the following: ?every "read" operation receives the most recent "write" operation results.

Consistency requires that each transaction in a database is valid according to the database's defined rules, and when the database changes state (some information has changed), such change is valid and does not corrupt the data. Each transaction moves the database from one valid state to another valid state. Consistency can be thought of as the following: every "read" operation receives the most recent "write" operation results.

Isolation means that you can "concurrently" (at the same time) run multiple transactions on a database, but the database will end up with a state that looks as though each operation had been run serially ( in a sequence, like a queue of operations). ?I personally think "Isolation" is not a very descriptive term for the concept, but I guess ACCD is less easy to say than ACID...

Isolation means that you can "concurrently" (at the same time) run multiple transactions on a database, but the database will end up with a state that looks as though each operation had been run serially ( in a sequence, like a queue of operations). I personally think "Isolation" is not a very descriptive term for the concept, but I guess ACCD is less easy to say than ACID...

Durability is the promise that once the data is stored in the database, it will remain so. ?It will be "persistent" - stored on disk and not in "memory". ?

Durability is the promise that once the data is stored in the database, it will remain so. It will be " persistent " - stored on disk and not in "memory".

Non-relational databases (Non-relational databases)

In contrast, a non-relational database has a less rigid, or, put another way, a more flexible structure to its data. ?The data typically is presented as "key-value" pairs. ?A simple way of representing this would be as an array (list) of "key-value" pair objects, for example:

In contrast, a non-relational database has a less rigid, or, put another way, a more flexible structure to its data. The data typically is presented as "key-value" pairs. A simple way of representing this would be as an array (list) of "key-value" pair objects, for example:

// baby names
[{ name: "Jacob",rank: ##,gender: "M",year: ####},{ name: "Isabella",rank: ##,gender: "F",year: ####},{//...},// ...
]

Non relational databases are also referred to as "NoSQL" databases, and offer benefits when you do not want or need to have consistently structured data.

Similar to the ACID properties, NoSQL database properties are sometimes referred to as BASE:

Basically Available which states that the system guarantees availability

Soft State mean means the state of the system may change over time, even without input

Eventual Consistency states that the system will become consistent over a (very short) period of time unless other inputs are received.

Since, at their core, these databases hold data in a hash-table-like structure, they are extremely fast, simple and easy to use, and are perfect for use cases like caching, environment variables, configuration files and session state etc. This flexibility makes them perfect for using in memory (e.g. Memcached) and also in persistent storage (e.g. DynamoDb).

Since, at their core, these databases hold data in a hash-table-like structure, they are extremely fast, simple and easy to use, and are perfect for use cases like caching, environment variables, configuration files and session state etc. This flexibility makes them perfect for using in memory (eg Memcached ) and also in persistent storage (eg DynamoDb ).

There are other "JSON-like" databases called document databases like the well-loved MongoDb, and at the core these are also "key-value" stores.

There are other "JSON-like" databases called document databases like the well-loved MongoDb , and at the core these are also "key-value" stores.

Database Indexing (Database Indexing)

This is a complicated topic so I will simply skim the surface for the purpose of giving you a high level overview of what you need for systems design interviews.

Imagine a database table with 100 million rows. ?This table is used mainly to look up one or two values in each record. To retrieve the values for a specific row you would need to iterate over the table. If it's the very last record that would take a long time!

Imagine a database table with 100 million rows. This table is used mainly to look up one or two values in each record. To retrieve the values for a specific row you would need to iterate over the table. If it's the very last record that would take a long time!

Indexing is a way of short cutting to the record that has matching values more efficiently than going through each row. Indexes are typically a data structure that is added to the database that is designed to facilitate fast searching of the database for those specific attributes (fields).

So if the census bureau has 120 million records with names and ages, and you most often need to retrieve lists of people belonging to an age group, then you would index that database on the age attribute.

Indexing is core to relational databases and is also widely offered on non-relational databases. The benefits of indexing are thus available in theory for both types of databases, and this is hugely beneficial to optimise lookup times.

Replication and Sharding (Replication and Sharding)

While these may sound like things out of a bio-terrorism movie, you're more likely to hear them everyday in the context of database scaling.

Replication means to duplicate (make copies of, replicate) your database. ?You may remember that when we discussed availability.

Replication means to duplicate (make copies of, replicate) your database. You may remember that when we discussed availability .

We had considered the benefits of having redundancy in a system to maintain high availability. Replication ensures redundancy in the database if one goes down. But it also raises the question of how to synchronize data across the replicas, since they're meant to have the same data. ?Replication on write and update operations to a database can happen synchronously (at the same time as the changes to the main database) or asynchronously . ?

We had considered the benefits of having redundancy in a system to maintain high availability. Replication ensures redundancy in the database if one goes down. But it also raises the question of how to synchronize data across the replicas, since they're meant to have the same data. Replication on write and update operations to a database can happen synchronously (at the same time as the changes to the main database) or asynchronously .

The acceptable time interval between synchronising the main and a replica database really depends on your needs - if you really need state between the two databases to be consistent then the replication needs to be rapid. ?You also want to ensure that if the write operation to the replica fails, the write operation to the main database also fails (atomicity).

The acceptable time interval between synchronising the main and a replica database really depends on your needs - if you really need state between the two databases to be consistent then the replication needs to be rapid. You also want to ensure that if the write operation to the replica fails, the write operation to the main database also fails (atomicity).

But what do you do when you've got so much data that simply replicating it may solve availability issues but does not solve throughput and latency issues (speed)? ?

But what do you do when you've got so much data that simply replicating it may solve availability issues but does not solve throughput and latency issues (speed)?

At this point you may want to consider "chunking down" your data, into "shards". Some people also call this partitioning your data (which is different from partitioning your hard drive!). ?

At this point you may want to consider "chunking down" your data, into "shards". Some people also call this partitioning your data (which is different from partitioning your hard drive!).

Sharding data breaks your huge database into smaller databases. ?You can work out how you want to shard your data depending on its structure. ?It could be as simple as every 5 million rows are saved in a different shard, or go for other strategies that best fit your data, needs and locations served.

Sharding data breaks your huge database into smaller databases. You can work out how you want to shard your data depending on its structure. It could be as simple as every 5 million rows are saved in a different shard, or go for other strategies that best fit your data, needs and locations served.

Section 9: Leader Election (Section 9: Leader Election)

Let's move back to servers again for a slightly more advanced topic. ?We already understand the principle of Availability, and how redundancy is one way to increase availability. ?We have also walked through some practical considerations when handling the routing of requests to clusters of redundant servers.

Let's move back to servers again for a slightly more advanced topic. We already understand the principle of Availability , and how redundancy is one way to increase availability. We have also walked through some practical considerations when handling the routing of requests to clusters of redundant servers.

But sometimes, with this kind of setup where multiple servers are doing much the same thing, there can arise situations where you need only one server to take the lead.

For example, you want to ensure that only one server is given the responsibility for updating some third party API because multiple updates from different servers could cause issues or run up costs on the third-party's side. ?

In this case you need to choose that primary server to delegate this update responsibility to. ?That process is called leader election. ?

In this case you need to choose that primary server to delegate this update responsibility to. That process is called leader election .

When multiple servers are in a cluster to provide redundancy, they could, amongst themselves, be configured to have one and only one leader. They would also detect when that leader server has failed, and appoint another one to take its place.

The principle is very simple, but the devil is in the details. ?The really tricky part is ensuring that the servers are "in sync" in terms of their data, state and operations.

The principle is very simple, but the devil is in the details. The really tricky part is ensuring that the servers are "in sync" in terms of their data, state and operations.

There is always the risk that certain outages could result in one or two servers being disconnected from the others, for example. ?In that case, engineers end up using some of the underlying ideas that are used in blockchain to derive consensus values for the cluster of servers. ?

There is always the risk that certain outages could result in one or two servers being disconnected from the others, for example. In that case, engineers end up using some of the underlying ideas that are used in blockchain to derive consensus values for the cluster of servers.

In other words, a consensus algorithm is used to give all the servers an "agreed on" value that they can all rely on in their logic when identifying which server is the leader.

Leader Election is commonly implemented with software like etcd, which is a store of key-value pairs that offers both high availability and strong consistency (which is valuable and an unusual combination) by using Leader Election itself and using a consensus algorithm. ?

Leader Election is commonly implemented with software like etcd , which is a store of key-value pairs that offers both high availability and strong consistency (which is valuable and an unusual combination) by using Leader Election itself and using a consensus algorithm.

So engineers can rely on etcd's own leader election architecture to produce leader election in their systems. This is done by storing in a service like etcd, a key-value pair that represents the current leader. ?

Since etcd is highly available and strongly consistent, that key-value pair can always be relied on by your system to contain the final "source of truth" server in your cluster is the current elected leader.

Section 10: Polling, Streaming, Sockets (Section 10: Polling, Streaming, Sockets)

In the modern age of continuous updates, push notifications, streaming content and real-time data, it is important to grasp the basic principles that underpin these technologies. ?To have data in your application updated regularly or instantly requires the use of one of the two following approaches.

In the modern age of continuous updates, push notifications, streaming content and real-time data, it is important to grasp the basic principles that underpin these technologies. To have data in your application updated regularly or instantly requires the use of one of the two following approaches.

Polling (Polling)

This one is simple. If you look at the wikipedia entry you may find it a bit intense. ?So instead take a look at its dictionary meaning, especially in the context of computer science. ?Keep that simple fundamental in mind.

This one is simple. If you look at the wikipedia entry you may find it a bit intense. So instead take a look at its dictionary meaning, especially in the context of computer science. Keep that simple fundamental in mind.

Polling is simply having your client "check" send a network request to your server and asking for updated data. ?These requests are typically made at regular intervals like 5 seconds, 15 seconds, 1 minute or any other interval required by your use case.

Polling is simply having your client "check" send a network request to your server and asking for updated data. These requests are typically made at regular intervals like 5 seconds, 15 seconds, 1 minute or any other interval required by your use case.

Polling every few seconds is still not quite the same as real-time, and also comes with the following downsides, especially if you have a million plus simultaneous users:

almost-constant network requests (not great for the client)
almost-constant network requests (not great for the client)
almost constant inbound requests (not great for the server loads - 1 million+ requests per second!)
almost constant inbound requests (not great for the server loads - 1 million+ requests per second!)

So polling rapidly is not really efficient or performant, and polling is best used in circumstances when small gaps in data updates is not a problem for your application.

For example, if you built an Uber clone, you may have the driver-side app send driver location data every 5 seconds, and your rider-side app poll for the driver's location every 5 seconds.

流媒體 (Streaming)

Streaming solves the constant polling problem. ?If constantly hitting the server is necessary, then it's better to use something called web-sockets.

Streaming solves the constant polling problem. If constantly hitting the server is necessary, then it's better to use something called web-sockets .

This is a network communication protocol that is designed to work over TCP. It opens a two-way dedicated channel (socket) between a client and server, kind of like an open hotline between two endpoints.

Unlike the usual TCP/IP communication, these sockets are "long-lived" so that its a single request to the server that opens up this hotline for the two-way transfer of data, rather than multiple separate requests. By long-lived, we meant that the socket connection between the machines will last until either side closes it, or the network drops.

You may remember from our discussion on IP, TCP and HTTP that these operate by sending "packets" of data, for each request-response cycle. ?Web-sockets mean that there is a single request-response interaction (not a cycle really if you think about it!) and that opens up the channel through which two-data is sent in a "stream".

You may remember from our discussion on IP, TCP and HTTP that these operate by sending "packets" of data, for each request-response cycle. Web-sockets mean that there is a single request-response interaction (not a cycle really if you think about it!) and that opens up the channel through which two-data is sent in a "stream".

The big difference with polling and all "regular" IP based communication is that whereas polling has the client making requests to the server for data at regular intervals ("pulling" data), in streaming, the client is "on standby" waiting for the server to "push" some data its way. The server will send out data when it changes, and the client is always listening for that. Hence, if the data change is constant, then it becomes a "stream", which may be better for what the user needs. ?

For example, while using collaborative coding IDEs, when either user types something, it can show up on the other, and this is done via web-sockets because you want to have real-time collaboration. ?It would suck if what I typed showed up on your screen after you tried to type the same thing or after 3 minutes of you waiting wondering what I was doing!

For example, while using collaborative coding IDEs , when either user types something, it can show up on the other, and this is done via web-sockets because you want to have real-time collaboration. It would suck if what I typed showed up on your screen after you tried to type the same thing or after 3 minutes of you waiting wondering what I was doing!

Or think of online, multiplayer games - that is a perfect use case for streaming game data between players!

To conclude, the use case determines the choice between polling and streaming. ?In general, you want to stream if your data is "real-time", and if it's OK to have a lag (as little as 15 seconds is still a lag) then polling may be a good option. But it all depends on how many simultaneous users you have and whether they expect the data to be instantaneous. A commonly used example of a streaming service is Apache Kafka.

To conclude, the use case determines the choice between polling and streaming. In general, you want to stream if your data is "real-time", and if it's OK to have a lag (as little as 15 seconds is still a lag) then polling may be a good option. But it all depends on how many simultaneous users you have and whether they expect the data to be instantaneous. A commonly used example of a streaming service is Apache Kafka .

Section 11: Endpoint Protection (Section 11: Endpoint Protection)

When you build large scale systems it becomes important to protect your system from too many operations, where such operations are not actually needed to use the system. Now that sounds very abstract. ?But think of this - how many times have you clicked furiously on a button thinking it's going to make the system more responsive? Imagine if each one of those button clicks pinged a server and the server tried to process them all! If the throughput of the system is low for some reason (say a server was struggling under unusual load) then each of those clicks would have made the system even slower because it has to process them all!

When you build large scale systems it becomes important to protect your system from too many operations, where such operations are not actually needed to use the system. Now that sounds very abstract. But think of this - how many times have you clicked furiously on a button thinking it's going to make the system more responsive? Imagine if each one of those button clicks pinged a server and the server tried to process them all! If the throughput of the system is low for some reason (say a server was struggling under unusual load) then each of those clicks would have made the system even slower because it has to process them all!

Sometimes it's not even about protecting the system. Sometimes you want to limit the operations because that is part of your service. For example, you may have used free tiers on third-party API services where you're only allowed to make 20 requests per 30 minute interval. if you make 21 or 300 requests in a 30 minute interval, after the first 20, that server will stop processing your requests.

That is called rate-limiting. Using rate-limiting, a server can limit the number of operations attempted by a client in a given window of time. A rate-limit can be calculated on users, requests, times, payloads, or other things. Typically, once the limit is exceeded in a time window, for the rest of that window the server will return an error.

Ok, now you might think that endpoint "protection" is an exaggeration. You're just restricting the users ability to get something out of the endpoint. ?True, but it is also protection when the user (client) is malicious - like say a bot that is smashing your endpoint. ?Why would that happen? Because flooding a server with more requests than it can handle is a strategy used by malicious folks to bring down that server, which effectively brings down that service. ?That's exactly what a Denial of Service (D0S) attack is.

Ok, now you might think that endpoint "protection" is an exaggeration. You're just restricting the users ability to get something out of the endpoint. True, but it is also protection when the user (client) is malicious - like say a bot that is smashing your endpoint. Why would that happen? Because flooding a server with more requests than it can handle is a strategy used by malicious folks to bring down that server, which effectively brings down that service. That's exactly what a Denial of Service (D0S) attack is.

While DoS attacks can be defended against in this way, rate-limiting by itself won't protect you from a sophisticated version of a DoS attack - a distributed DoS. Here distribution simply means that the attack is coming from multiple clients that seem unrelated and there is no real way to identify them as being controlled by the single malicious agent. ?Other methods need to be used to protect against such coordinated, distributed attacks.

While DoS attacks can be defended against in this way, rate-limiting by itself won't protect you from a sophisticated version of a DoS attack - a distributed DoS. Here distribution simply means that the attack is coming from multiple clients that seem unrelated and there is no real way to identify them as being controlled by the single malicious agent. Other methods need to be used to protect against such coordinated, distributed attacks.

But rate-limiting is useful and popular anyway, for less scary use-cases, like the API restriction one I mentioned. ?Given how rate-limiting works, since the server has to first check the limit conditions and enforce them if necessary, you need to think about what kind of data structure and database you'd want to use to make those checks super fast, so that you don't slow down processing the request if it's within allowed limits. Also, if you have it in-memory within the server itself, then you need to be able to guarantee that all requests from a given client will come to that server so that it can enforce the limits properly. ?To handle situations like this it's popular to use a separate Redis service that sits outside the server, but holds the user's details in-memory, and can quickly determine whether a user is within their permitted limits.

But rate-limiting is useful and popular anyway, for less scary use-cases, like the API restriction one I mentioned. Given how rate-limiting works, since the server has to first check the limit conditions and enforce them if necessary, you need to think about what kind of data structure and database you'd want to use to make those checks super fast, so that you don't slow down processing the request if it's within allowed limits. Also, if you have it in-memory within the server itself, then you need to be able to guarantee that all requests from a given client will come to that server so that it can enforce the limits properly. To handle situations like this it's popular to use a separate Redis service that sits outside the server, but holds the user's details in-memory, and can quickly determine whether a user is within their permitted limits.

Rate limiting can be made as complicated as the rules you want to enforce, but the above section should cover the fundamentals and most common use-cases.

Section 12: Messaging & Pub-Sub (Section 12: ?Messaging & Pub-Sub)

When you design and build large-scale and distributed systems, for that system to work cohesively and smoothly, it is important to exchange information between the components and services that make up the system. But as we have seen before, systems that rely on networks suffer from the same weakness as networks - they are fragile. Networks fail and its not an infrequent occurrence. ?When networks fail, components in the system are not able to communicate may degrade the system (best case) or cause the system to fail altogether (worst case). ?So distributed systems need robust mechanisms to ensure that the communication continues or recovers where it left off, even if there is an "arbitrary partition" (i.e. failure) between components in the system.

When you design and build large-scale and distributed systems , for that system to work cohesively and smoothly, it is important to exchange information between the components and services that make up the system. But as we have seen before, systems that rely on networks suffer from the same weakness as networks - they are fragile. Networks fail and its not an infrequent occurrence. When networks fail, components in the system are not able to communicate may degrade the system (best case) or cause the system to fail altogether (worst case). So distributed systems need robust mechanisms to ensure that the communication continues or recovers where it left off, even if there is an "arbitrary partition" (ie failure) between components in the system.

Imagine, as an example, that you're booking airline tickets. You get a good price, choose your seats, confirm the booking and you've even paid using your credit card. ?Now you're waiting for your ticket PDF to arrive in your inbox. ?You wait, and wait, and it never comes. ?Somewhere, there was a system failure that didn't get handled or recover properly. ?A booking system will often connect with airline and pricing APIs to handle the actual flight selection, fare summary, date and time of flight etc. ?All that gets done while you click through the site's booking UI. But it doesn't have to send you the PDF of the tickets until a few minutes later. Instead the UI can simply confirm that your booking is done, and you can expect the tickets in your inbox shortly. That's a reasonable and common user experience for bookings because the moment of paying and the receipt of the tickets does not have to be simultaneous - the two events can be asynchronous. ?Such a system would need messaging to ensure that the service (server endpoint) that ?asynchronously generates the PDF gets notified of a confirmed, paid-for booking, and all the details, and then the PDF can be auto-generated and emailed to you. ?But if that messaging system fails, the email service would never know about your booking and no ticket would get generated.

Imagine, as an example, that you're booking airline tickets. You get a good price, choose your seats, confirm the booking and you've even paid using your credit card. Now you're waiting for your ticket PDF to arrive in your inbox. You wait, and wait, and it never comes. Somewhere, there was a system failure that didn't get handled or recover properly. A booking system will often connect with airline and pricing APIs to handle the actual flight selection, fare summary, date and time of flight etc. All that gets done while you click through the site's booking UI. But it doesn't have to send you the PDF of the tickets until a few minutes later. Instead the UI can simply confirm that your booking is done, and you can expect the tickets in your inbox shortly. That's a reasonable and common user experience for bookings because the moment of paying and the receipt of the tickets does not have to be simultaneous - the two events can be asynchronous. Such a system would need messaging to ensure that the service (server endpoint) that asynchronously generates the PDF gets notified of a confirmed, paid-for booking, and all the details, and then the PDF can be auto-generated and emailed to you. But if that messaging system fails, the email service would never know about your booking and no ticket would get generated.

Publisher / Subscriber Messaging

This is a very popular paradigm (model) for messaging. The key concept is that publishers 'publish' a message and a subscriber subscribes to messages. ?To give greater granularity, messages can belong to a certain "topic" which is like a category. ?These topics are like dedicated "channels" or pipes, where each pipe exclusives handles messages belonging to a specific topic. ?Subscribers choose which topic they want to subscribe to and get notified of messages in that topic. ?The advantage of this system is that the publisher and the subscriber can be completely de-coupled - i.e. they don't need to know about each other. ?The publisher announces, and the subscriber listens for announcements for topics that it is on the lookout for.

This is a very popular paradigm (model) for messaging. The key concept is that publishers 'publish' a message and a subscriber subscribes to messages. To give greater granularity, messages can belong to a certain "topic" which is like a category. These topics are like dedicated "channels" or pipes, where each pipe exclusives handles messages belonging to a specific topic. Subscribers choose which topic they want to subscribe to and get notified of messages in that topic. The advantage of this system is that the publisher and the subscriber can be completely de-coupled - ie they don't need to know about each other. The publisher announces, and the subscriber listens for announcements for topics that it is on the lookout for.

A server is often the publisher of messages and there are usually several topics (channels) that gets published to. ?The consumer of a specific topic subscribes to those topics. There is no direct communication between the server (publisher) and the subscriber (could be another server). The only interaction is between publisher and topic, and topic and subscriber.

A server is often the publisher of messages and there are usually several topics (channels) that gets published to. The consumer of a specific topic subscribes to those topics. There is no direct communication between the server (publisher) and the subscriber (could be another server). The only interaction is between publisher and topic, and topic and subscriber.

The messages in the topic are just data that needs to be communicated, and can take on whatever forms you need. So that gives you four players in Pub/Sub: Publisher, Subscriber, Topics and Messages.

Better than a database (Better than a database)

So why bother with this? Why not just persist all data to a database and consume it directly from there? Well, you need a system to queue up the messages because ?each message corresponds to a task that needs to be done based on that message's data. So in our ticketing example, if a 100 people make a booking in 35 minutes, putting all that in the database doesn't solve the problem of emailing those 100 people. It just stores a 100 transactions. ? Pub/Sub systems handle the communication, the task sequencing and the messages get persisted in a database. ?So the system can offer useful features like "at least once" delivery (messages won't be lost), persistent storage, ordering of messages, "try-again", "re-playability" of messages etc. Without this system, just storing the messages in the database will not help you ensure that the message gets delivered (consumed) and acted upon to successfully complete the task. ?

So why bother with this? Why not just persist all data to a database and consume it directly from there? Well, you need a system to queue up the messages because each message corresponds to a task that needs to be done based on that message's data. So in our ticketing example, if a 100 people make a booking in 35 minutes, putting all that in the database doesn't solve the problem of emailing those 100 people. It just stores a 100 transactions. Pub/Sub systems handle the communication, the task sequencing and the messages get persisted in a database. So the system can offer useful features like "at least once" delivery (messages won't be lost), persistent storage, ordering of messages, "try-again", "re-playability" of messages etc. Without this system, just storing the messages in the database will not help you ensure that the message gets delivered (consumed) and acted upon to successfully complete the task.

Sometimes the same message may get consumed more than once by a subscriber - typically because the network dropped out momentarily, and though the subscriber consumed the message, it didn't let the publisher know. So the publisher will simply re-send it to the subscriber. ?That's why the guarantee is "at least once" and not "once and only once". This is unavoidable in distributed systems because networks are inherently unreliable. ?This can raise complications, where the message triggers an operation on the subscriber's side, and that operation could change things in the database (change state in the overall application). ?What if a single operation gets repeated multiple times, and each time the application's state changes?

Sometimes the same message may get consumed more than once by a subscriber - typically because the network dropped out momentarily, and though the subscriber consumed the message, it didn't let the publisher know. So the publisher will simply re-send it to the subscriber. That's why the guarantee is "at least once" and not "once and only once". This is unavoidable in distributed systems because networks are inherently unreliable. This can raise complications, where the message triggers an operation on the subscriber's side, and that operation could change things in the database (change state in the overall application). What if a single operation gets repeated multiple times, and each time the application's state changes?

Controlling Outcomes - one or many outcomes? (Controlling Outcomes - one or many outcomes?)

The solution to this new problem is called idempotency - which is a concept that is important but not intuitive to grasp the first few times you examine it. It is a concept that can appear complex (especially if you read the wikipedia entry), so for the current purpose, here is a user-friendly simplification from StackOverflow:

In computing, an idempotent operation is one that has no additional effect if it is called more than once with the same input parameters.
In computing, an idempotent operation is one that has no additional effect if it is called more than once with the same input parameters.

So when a subscriber processes a message two or three times, the overall state of the application is exactly what it was after the message was processed the first time. If, for example, at the end of booking your flight tickets and after you entered your credit card details, you clicked on "Pay Now" three times because the system was slow ... you would not want to pay 3X the ticket price right? You need idempotency to ensure that each click after the first one doesn't make another purchase and charge your credit card more than once. In contrast, you can post an identical comment on your best friend's newsfeed N number of times. They will all show up as separate comments, and apart from being annoying, that's not actually wrong. Another example is offering "claps" on Medium posts - each clap is meant to increment the number of claps, not be one and only one clap. These latter two examples do not require idempotency, but the payment example does.

There are many flavours of messaging systems, and the choice of system is driven by the use-case to be solved for. ?Often, people will refer to "event based" architecture which means that the system relies on messages about "events" (like paying for tickets) to process operations (like emailing the ticket). ?The really commonly talked about services are Apache Kafka, RabbitMQ, Google Cloud Pub/Sub, AWS SNS/SQS.

There are many flavours of messaging systems, and the choice of system is driven by the use-case to be solved for. Often, people will refer to "event based" architecture which means that the system relies on messages about "events" (like paying for tickets) to process operations (like emailing the ticket). The really commonly talked about services are Apache Kafka, RabbitMQ, Google Cloud Pub/Sub, AWS SNS/SQS.

Section 13: Smaller Essentials (Section 13: Smaller Essentials)

記錄中 (Logging)

Over time your system will collect a lot of data. ?Most of this data is extremely useful. It can give you a view of the health of your system, its performance and problems. It can also give you valuable insight into who uses your system, how they use it, how often, which parts get used more or less, and so on. ?

Over time your system will collect a lot of data. Most of this data is extremely useful. It can give you a view of the health of your system, its performance and problems. It can also give you valuable insight into who uses your system, how they use it, how often, which parts get used more or less, and so on.

This data is valuable for analytics, performance optimization and product improvement. It is also extremely valuable for debugging, not just when you log to your console during development, but in actually hunting down bugs in your test and production environments. So logs help in traceability and audits too. ?

The key trick to remember when logging is to view it as a sequence of consecutive events, which means the data becomes time-series data, and the tools and databases you use should be specifically designed to help work with that kind of data.

監控方式 (Monitoring)

This is the next steps after logging. ?It answers the question of "What do I do with all that logging data?". You monitor and analyze it. ?You build or use tools and services that parse through that data and present you with dashboards or charts or other ways of making sense of that data in a human-readable way.

This is the next steps after logging. It answers the question of "What do I do with all that logging data?". You monitor and analyze it. You build or use tools and services that parse through that data and present you with dashboards or charts or other ways of making sense of that data in a human-readable way.

By storing the data in a specialized database designed to handle this kind of data (time-series data) you can plug in other tools that are built with that data structure and intention in mind.

Alerting (Alerting)

When you are actively monitoring you should also put a system in place to alert you of significant events. Just like having an alert for stock prices going over a certain ceiling or below a certain threshold, certain metrics that you're watching may warrant an alert being sent if they go too high or too low. Response times (latency) or errors and failures are good ones to set up alerting for if they go above an "acceptable" level.

The key to good logging and monitoring is to ensure your data is fairly consistent over time, as working with inconsistent data could result in missing fields that then break the analytical tools or reduce the benefits of the logging.

資源資源 (Resources)

As promised, some useful resources are as follows:

A fantastic Github repo full of concepts, diagrams and study prep
A fantastic Github repo full of concepts, diagrams and study prep
Tushar Roy's introduction to Systems Design
Tushar Roy's introduction to Systems Design
Gaurav Sen's YouTube playlist
Gaurav Sen's YouTube playlist
SQL vs NoSQL
SQL vs NoSQL

I hope you enjoyed this long-form guide!

You can ask me questions on Twitter.

You can ask me questions on Twitter .

Postscript for freeCodeCamp students

Postscript f or f reeCodeCamp students

I really, truly believe your most precious resources are your time, effort and money. Of these, the single most important resource is time, because the other two can be renewed and recovered. So if you’re going to spend time on something make sure it gets you closer to this goal.

I really, truly believe your most precious resources are your time, effort and money. Of these, the single most important resource is time, because the other two can be renewed and recovered. So if you're going to spend time on something make sure it gets you closer to this goal.

With that in mind, if you want to invest 3 hours with me to find your shortest path to learning to code (especially if you’re a career changer, like me), then head to my course site and use the form there sign up (not the popup!). If you add the words “I LOVE CODE” to the message, I will know you’re a freeCodeCamp reader, and I will send you a promo code, because just like you, freeCodeCamp gave me a solid start.

With that in mind, if you want to invest 3 hours with me to find your shortest path to learning to code (especially if you're a career changer, like me), then head to my course site and use the form there sign up (not the popup!). If you add the words “I LOVE CODE” to the message, I will know you're a freeCodeCamp reader, and I will send you a promo code, because just like you, freeCodeCamp gave me a solid start.

Also if you would like to learn more, check out ?episode 53 of the ?freeCodeCamp podcast, where Quincy (founder of FreeCodeCamp) and I share our experiences as career changers that may help you on your journey. You can also access the podcast on iTunes, Stitcher, and Spotify.

Also if you would like to learn more, check out episode 53 of the freeCodeCamp podcast , where Quincy (founder of FreeCodeCamp) and I share our experiences as career changers that may help you on your journey. You can also access the podcast on iTunes , Stitcher , and Spotify .