004:神秘的數組初始化_使容器神秘化101：面向初學者的深入研究容器技術

004:神秘的數組初始化

by Will Wang

王Will

介紹 (Introduction)

Regardless of whether you are a student in school, a developer at some company, or a software enthusiast, chances are you heard of containers. You may have also heard that containers are lightweight virtual machines, but what does that really mean, how exactly do containers work, and why are they so important?

無論您是學校的學生，某公司的開發人員還是軟件愛好者，您都有可能聽說過容器。您可能還聽說過容器是輕量級虛擬機，但這實際上是什么意思，容器如何工作，它們為什么如此重要？

This story serves as a look into containers, their key great technical ideas, and the applications. I won’t assume any prior knowledge in this field other than a basic understanding of computer science.

這個故事是對容器，其關鍵的重要技術思想以及應用程序的考察。除了對計算機科學的基本了解之外，我不會假定在該領域有任何先驗知識。

內核和操作系統 (The Kernel and the OS)

Your laptop, along with every other computer, is built on top of some pieces of hardware like the CPU, persistent storage (disk drive, SSD), memory, network card, etc.

您的筆記本電腦以及其他所有計算機都建立在某些硬件之上，例如CPU，持久性存儲(磁盤驅動器，SSD)，內存，網卡等。

To interact with this hardware, a piece of software in the operating system called the kernel serves as the bridge between the hardware and the rest of the system. The kernel is responsible for scheduling processes (programs) to run, managing devices (reading and writing addresses on disk and memory), and more.

為了與此硬件進行交互，操作系統中的一個稱為內核的軟件充當了硬件與系統其余部分之間的橋梁。內核負責調度要運行的進程 (程序)，管理設備(在磁盤和內存上讀取和寫入地址)等。

The rest of the operating system serves to boot and manage the user space, where user processes are run, and will constantly interact with the kernel.

其余的操作系統用于引導和管理用戶空間(運行用戶進程)，并且將不斷與內核進行交互。

虛擬機 (The Virtual Machine)

So you have a computer that runs MacOS and an application that is built to run on Ubuntu. Hmmm… One common solution is to boot up a virtual machine on your MacOS computer that runs Ubuntu and then run your program there.

因此，您有一臺運行MacOS的計算機和一個可以在Ubuntu上運行的應用程序。嗯……一種常見的解決方案是在運行Ubuntu的MacOS計算機上啟動虛擬機，然后在其中運行程序。

A virtual machine is comprised of some level of hardware and kernel virtualization on which runs a guest operating system. A piece of software called a hypervisor creates the virtualized hardware which may include the virtual disk, virtual network interface, virtual CPU, and more. Virtual machines also include a guest kernel that can talk to this virtual hardware.

虛擬機由運行客戶操作系統的某種級別的硬件和內核虛擬化組成。稱為管理程序的軟件會創建虛擬化的硬件，其中可能包括虛擬磁盤，虛擬網絡接口，虛擬CPU等。虛擬機還包括可以與該虛擬硬件通信的來賓內核。

The hypervisor can be hosted, which means it is some software that runs on the Host OS (MacOS) as in the example. It can also be bare metal, running directly on the machine hardware (replacing your OS). Either way, the hypervisor approach is considered heavy weight as it requires virtualizing multiple parts if not all of the hardware and kernel.

可以托管虛擬機管理程序，這意味著它是某些在示例示例中在主機OS(MacOS)上運行的軟件。它也可以是裸機，直接在機器硬件上運行(替換您的OS)。無論哪種方式，系統管理程序方法都被視為繁重的工作，因為它需要虛擬化多個部分(如果不是全部的硬件和內核)。

When there needs to be multiple isolated groups on the same machine, running a VM for each of these groups is way too heavy and wasteful of resources to be a good approach.

當同一臺計算機上需要有多個隔離的組時，為這些組中的每個組運行VM太繁瑣且浪費資源，因此不是一個好的方法。

VMs require hardware virtualization for machine level isolation whereas containers operate on isolation within the same operation system. The overhead difference becomes really apparent as the number of isolated spaces increase. A regular laptop can run tens of containers but can struggle to run even one VM well.

VM需要硬件虛擬化以實現計算機級別的隔離，而容器則在同一操作系統內以隔離方式運行。隨著隔離空間數量的增加，開銷的差異變得非常明顯。一臺普通的筆記本電腦可以運行數十個容器，但是即使要很好地運行一個虛擬機，也很難。

小組 (cgroups)

In 2006, engineers at Google invented the Linux “control groups”, abbreviated as cgroups. This is a feature of the Linux kernel that isolates and controls the resource usage for user processes.

2006年，Google的工程師發明了Linux“控制組”，縮寫為cgroup 。這是Linux內核的功能，可隔離和控制用戶進程的資源使用情況。

These processes can be put into namespaces, essentially collections of processes that share the same resource limitations. A computer can have multiple namespaces, each with the resource properties enforced by the kernel.

這些進程可以放在命名空間中 ，本質上是共享相同資源限制的進程的集合。一臺計算機可以有多個命名空間，每個命名空間都具有內核強制執行的資源屬性。

The resource allocation per namespace can be managed in order to limit the amount of the overall CPU, RAM, etc that a set of processes can use. For example, a background log aggregation application will probably need to have its resources limit in order to not accidentally overwhelm the actual server it’s logging.

可以管理每個名稱空間的資源分配，以限制一組進程可以使用的總體CPU，RAM等的數量。例如，后臺日志聚合應用程序可能需要具有其資源限制，以免意外使它正在記錄的實際服務器不堪重負。

While not an original feature, cgroups in Linux were eventually reworked to include a feature called namespace isolation. The idea of namespace isolation itself is not new, and Linux already had many kinds of namespace isolation. One common example is process isolation, which separates each individual process and prevents such things like shared memory.

雖然不是原始功能，但Linux中的cgroup最終經過了重新設計，以包含稱為名稱空間隔離的功能。名稱空間隔離的思想本身并不是什么新鮮事物，Linux已經有了許多種名稱空間隔離。一個常見的示例是進程隔離，它隔離每個單獨的進程并防止諸如共享內存之類的事情。

Cgroup isolation is a higher level of isolation that makes sure processes within a cgroup namespace are independent of processes in other namespaces. A few important namespace isolation features are outlined below and pave the foundation for the isolation we expect from containers.

Cgroup隔離是更高級別的隔離，可確保cgroup命名空間中的進程獨立于其他命名空間中的進程。下面概述了一些重要的名稱空間隔離功能，這些功能為我們期望與容器隔離提供了基礎。

PID (Process Identifier) Namespaces: this ensures that processes within one namespace are not aware of process in other namespaces.
PID(進程標識符)命名空間：這可確保一個命名空間中的進程不知道其他命名空間中的進程。
Network Namespaces: Isolation of the network interface controller, iptables, routing tables, and other lower level networking tools.
網絡命名空間：網絡接口控制器，iptables，路由表和其他較低級別的網絡工具的隔離。
Mount Namespaces: Filesystems are mounted, so that the file system scope of a namespace is limited to only the directories mounted.
掛載命名空間：掛載文件系統，以便命名空間的文件系統范圍僅限于掛載的目錄。
User Namespaces: Limits users within a namespace to only that namespace and avoids user ID conflicts across namespaces.
用戶命名空間：將命名空間中的用戶限制為僅該命名空間，并避免跨命名空間的用戶ID沖突。

To put it simply, each namespace would appear to be its own machine to the processes within it.

簡單來說，每個名稱空間對于其中的進程而言似乎都是其自己的機器。

Linux容器 (Linux Containers)

Linux cgroups paved the way for a technology called linux containers (LXC). LXC was really the first major implementation of what we know today to be a container, taking advantage of cgroups and namespace isolation to create virtual environment with separate process and networking space.

Linux cgroup為稱為linux容器 (LXC)的技術鋪平了道路。 LXC實際上是我們今天所知的第一個主要實現，它是一個容器，它利用cgroup和名稱空間隔離來創建具有獨立進程和網絡空間的虛擬環境。

In a sense, this allows for independent and isolated user spaces. The idea of containers follows directly from LXC. In fact, earlier versions of Docker were built directly on top of LXC.

從某種意義上講，這允許獨立且隔離的用戶空間 。容器的思想直接來自LXC。實際上，早期版本的Docker是直接在LXC之上構建的。

碼頭工人 (Docker)

Docker is the most widely used container technology and really what most people mean when they refer to containers. While there are other open source container techs (like rkt by CoreOS) and large companies that build their own container engine (like lmctfy at Google), Docker has become the industry standard for containerization. It is still built on the cgroups and namespacing provided by the Linux kernel and recently Windows as well.

Docker是使用最廣泛的容器技術，實際上，大多數人指的是容器。盡管還有其他開源容器技術(例如CoreOS的rkt )和大型公司構建自己的容器引擎(例如Google的lmctfy )，但Docker已成為容器化的行業標準。它仍然基于Linux內核以及最近的Windows提供的cgroup和命名空間。

A Docker container is made up of layers of images, binaries packed together into a single package. The base image contains the operating system of the container, which can be different from the OS of the host.

Docker容器由圖像層組成，二進制文件打包在一起形成一個包。基本映像包含容器的操作系統，該操作系統可能與主機的OS不同。

The OS of the container is in the form an image. This is not the full operating system as on the host, and the difference is that the image is just the file system and binaries for the OS while the full OS includes the file system, binaries, and the kernel.

容器的操作系統采用映像形式。這不是主機上的完整操作系統，區別在于映像只是OS的文件系統和二進制文件，而完整OS包括文件系統，二進制文件和內核。

On top of the base image are multiple images that each build a portion of the container. For example, on top of the base image may be the image that contains the apt-get dependencies. On top of that may be the image that contains the application binary, and so on.

在基礎圖像之上是多個圖像，每個圖像都構成了容器的一部分。例如，在基本映像的頂部可能是包含apt-get依賴項的映像。最重要的是包含應用程序二進制文件的映像，依此類推。

The cool part is if there are two containers with the image layers a, b, c and a, b, d, then you only need to store one copy of each image layer a, b, c, d both locally and in the repository. This is Docker’s union file system.

最酷的部分是，如果有兩個容器分別具有圖像層a, b, c和a, b, d ，那么您只需要在本地和存儲庫中存儲每個圖像層a, b, c, d一個副本。這是Docker的聯合文件系統 。

Each image, identified by a hash, is just one of many possible layers of images that make up a container. However a container is identified only by its top level image, which has references to parent images. Two top level images (Image 1 and Image 2) shown here share the first three layers. Image 2 has two additional configuration related layers, but shares the same parent images as Image 1.

每個由哈希標識的圖像只是構成容器的許多可能的圖像層之一。但是，容器僅由其頂層圖像標識，該頂層圖像引用了父圖像。此處顯示的兩個頂級圖像(圖像1和圖像2)共享前三層。映像2具有兩個附加的與配置相關的層，但與映像1共享相同的父映像。

When a container is booted, the image and its parent images are downloaded from the repo, the cgroup and namespaces are created, and the image is used to create a virtual environment. From within the container, the files and binaries specified in the image appear to be the only files in the entire machine. Then the container’s main process is started and the container is considered alive.

啟動容器時，將從存儲庫中下載映像及其父映像，創建cgroup和名稱空間，然后使用該映像創建虛擬環境。從容器內部，映像中指定的文件和二進制文件似乎是整個計算機中唯一的文件。然后，啟動容器的主要過程，并認為該容器處于活動狀態。

Docker has some other really really cool features, such as copy on write, volumes (shared file systems between containers), the docker daemon (manages containers on a machine), version controlled repositories (like Github for containers), and more. To learn more about them and see some practical examples of how to use Docker, this Medium article is extremely useful.

Docker還有其他一些非常酷的功能，例如寫入時復制，卷(容器之間的共享文件系統)，docker守護進程(管理機器上的容器)，版本控制的存儲庫(例如Github用于容器)等等。要了解有關它們的更多信息并查看一些有關如何使用Docker的實際示例，這篇中型文章非常有用。

為什么選擇集裝箱 (Why Containers)

Aside from process isolation, containers have many other beneficial properties.

除了過程隔離之外，容器還具有許多其他有益的特性。

The container serves as a self isolated unit that can run anywhere that supports it. And in each of these instances, the container itself will be exactly identical. It won’t matter if the host OS is CentOS, Ubuntu, MacOS, or even something non UNIX like Windows — from within the container the OS will be whatever OS the container specified. Thus you can be sure the container you built on your laptop will also run on the company’s servers.

容器是一個自我隔離的單元，可以在支持它的任何位置運行。在每種情況下，容器本身都是完全相同的。主機操作系統是CentOS，Ubuntu，MacOS還是什至是非UNIX之類的Windows，都無所謂-從容器內部，操作系統將是容器指定的任何操作系統。因此，您可以確定在筆記本電腦上構建的容器也將在公司的服務器上運行。

The container also acts as a standardized unit of work or compute. A common paradigm is for each container to run a single web server, a single shard of a database, or a single Spark worker, etc. Then to scale an application, you simply need to scale the number of containers.

容器還充當工作或計算的標準化單位。一個常見的范例是，每個容器運行單個Web服務器，數據庫的單個碎片或單個Spark工作者等。然后要擴展應用程序，只需要擴展容器的數量即可。

In this paradigm, each container is given a fixed resource configuration (CPU, RAM, # of threads, etc) and scaling the application requires scaling just the number of containers instead of the individual resource primitives. This provides a much easier abstraction for engineers when applications need to be scaled up or down.

在此范例中，為每個容器分配了固定的資源配置(CPU，RAM，線程數等)，并且縮放應用程序僅需要縮放容器的數量，而不是單個資源原語。當需要按比例放大或縮小應用程序時，這為工程師提供了更容易的抽象。

Containers also serve as a great tool to implement micro service architecture, where each microservice is just a set of co-operating containers. For example the Redis micro service can be implemented with a single primary container and multiple replica containers.

容器也是實現微服務體系結構的好工具，其中每個微服務只是一組協作容器。例如，Redis微服務可以通過單個主容器和多個副本容器來實現。

This (micro)service orientated architecture has some very important properties that make it easy for engineering teams to create and deploy applications (see my earlier article for more details).

這種面向(微)服務的體系結構具有一些非常重要的屬性，這些屬性使工程團隊可以輕松創建和部署應用程序(有關更多詳細信息，請參閱我之前的文章 )。

編排 (Orchestration)

Ever since the time of linux containers, users have tried to deploy large scale applications over many virtual machines where each process runs in its own container. Doing this required being able to efficiently deploy tens to thousands of containers across potentially hundreds of virtual machines and manage their networking, file systems, resources, etc. Docker today makes this a little easier as it exposes abstractions to define container networking, volumes for file systems, resource configurations, etc.

自從使用Linux容器以來，用戶一直試圖在許多虛擬機上部署大型應用程序，其中每個進程都在其自己的容器中運行。為此，必須能夠在潛在的數百個虛擬機中有效地部署數以萬計的容器，并管理其網絡，文件系統，資源等。如今，Docker公開了定義容器網絡，文件卷的抽象概念，使這一過程變得容易一些系統，資源配置等

However a tool is still needed to:

但是，仍然需要一種工具來：

actually take a specification and assign containers to machines (scheduling)
實際采用規格并將容器分配給機器(計劃)
actually boot the specified containers on the machines through Docker
實際上通過Docker引導機器上的指定容器
deal with upgrades/rollbacks/the constantly changing nature of the system
處理升級/回滾/系統不斷變化的性質
respond to failures like container crashes
應對容器崩潰等故障
and create cluster resources like service discovery, inter VM networking, cluster ingress/egress, etc.
并創建群集資源，例如服務發現，VM間網絡連接，群集入口/出口等。

This set of problems relates to the orchestration of a distributed system built on top of a set of (possibly transient or constantly changing) containers, and people have built some really miraculous systems to solve this problem.

這一系列問題與在一組(可能是瞬態或不斷變化的)容器之上構建的分布式系統的編排有關，人們已經構建了一些真正神奇的系統來解決此問題。

In my next story I will talk in depth about the implementation of Kubernetes, the major open source orchestrator, along with two equally important but lesser known ones, Mesos and Borg.

在我的下一個故事中，我將深入探討主要的開源編排Kubernetes的實現，以及同等重要但鮮為人知的兩個Mesos和Borg。

This story is part of a series. I am an undergrad at UC Berkeley. My research is in distributed systems and I am advised by Scott Shenker.

這個故事是系列的一部分。我是加州大學伯克利分校的本科生。我的研究是在分布式系統上，我得到了Scott Shenker的建議。