A brief guide to building an app to serve a natural language processing model, containerizing it and deploying it.

構建用于服務自然語言處理模型，將其容器化和部署的應用程序的簡要指南。

By: Edward Krueger and Douglas Franklin.

作者： 愛德華·克魯格 ( Edward Krueger)和道格拉斯·富蘭克林 ( Douglas Franklin) 。

If you need help building an NLP pipeline or evaluating models check out our last article. We covered some NLP basics and how to build an NLP pipeline with Scikit-Learn. Then we evaluated some model metrics and decided on the best model for our problem and data.

如果您需要建立NLP管道或評估模型的幫助，請查看我們的上一篇文章 。 我們介紹了一些NLP基礎知識，以及如何使用Scikit-Learn構建NLP管道。 然后，我們評估了一些模型指標，并為我們的問題和數據確定了最佳模型。

Be sure to check out the README and code in our GitHub repository instructions on setting up this app locally with Docker!

請務必查看自述文件和GitHub存儲庫中的代碼，以了解如何使用Docker在本地設置此應用程序！

Before deploying our model to the cloud, we need to build an app to serve our model.

在將我們的模型部署到云之前，我們需要構建一個應用程序來服務我們的模型。

構建應用 (Building the App)

The application to serve this model is simple. We just need to import our model into the app, receive a POST request and return the model’s response to that POST.

服務于該模型的應用程序很簡單。 我們只需要將模型導入應用程序，接收POST請求，然后將模型的響應返回給該POST。

Here is the app code.

這是應用程序代碼。

Notice that the data_dict is the Python dictionary corresponding to the payload sent via the POST request. From this dictionary we can extract the text to be categorized. This means our POSTs need the same key. So we are going to send JSONs with the {key: value} being {"text": "message") .

請注意， data_dict是與通過POST請求發送的有效負載相對應的Python字典。 從這本字典中，我們可以提取要分類的文本。 這意味著我們的POST需要相同的密鑰。 因此，我們將發送{key: value}為{"text": "message") JSON。

Since the Scikit-Learn pipeline expects a list of strings we have to wrap the text as a list. Also, when receiving the result from the model’s .predict method, we receive a list with a single element and unpack it to access the prediction.

由于Scikit-Learn管道需要字符串列表，因此我們必須將文本包裝為列表。 另外，從模型的.predict方法接收結果時，我們會收到一個包含單個元素的列表，并將其解包以訪問預測。

Once we have the app running locally without bugs, we are ready to make some changes to our repository to prepare for deployment.

一旦我們使應用程序在本地運行且沒有錯誤，就可以對存儲庫進行一些更改以準備部署。

Keep in mind we are using the package and environment manager pipenv to handle our app’s dependencies. If you are not familiar with virtual environment and package management, you may need to install this and set up a virtual environment using the Pipfile in the Github repository. Check out this article for help doing that!

請記住，我們正在使用程序包和環境管理器pipenv處理我們應用程序的依賴項。 如果您不熟悉虛擬環境和軟件包管理，則可能需要安裝此文件并使用Gi??thub存儲庫中的Pipfile設置虛擬環境。 請查看本文以獲取幫助！

容器化應用 (Containerizing the App)

We need to make some final changes to our project files in preparation for deployment. In our project, we’ve used pipenv and pipenv-to-requirements to handle dependencies and generate a requirements.txt. All you'll need for your Docker container is the requirements.txt file.

我們需要對我們的項目文件進行一些最終更改，以準備進行部署。 在我們的項目中，我們使用pipenv和pipenv-to-requirements來處理依賴關系并生成requirements.txt 。 您的Docker容器所需的全部是requirements.txt文件。

Be sure to git add the Dockerfile you made earlier to your repository.

確保將之前創建的Dockerfile添加到倉庫中。

Here is the link to our Pipfile for dependencies on this project.

這是指向該項目依賴項的Pipfile的鏈接。

Docker和Dockerfiles (Docker and Dockerfiles)

Before we get the app running in the cloud, we must first Dockerize it. Check out our README in the GitHub repository for instructions on setting up this app locally with Docker.

在使應用程序在云中運行之前，我們必須首先對其進行Dockerize。 請查看GitHub存儲庫中的自述文件，以獲取有關使用Docker在本地設置此應用程序的說明。

Docker is the best way to put apps into production. Docker uses a Dockerfile to build a container. The built container is stored in Google Container Registry were it can be deployed. Docker containers can be built locally and will run on any system running Docker.

Docker是將應用程序投入生產的最佳方式。 Docker使用Dockerfile來構建容器。 構建的容器可以存儲在Google Container Registry中。 Docker容器可以在本地構建，并且可以在運行Docker的任何系統上運行。

Here is the Dockerfile we used for this project:

這是我們用于該項目的Dockerfile：

The first line of every Dockerfile begins with FROM. This is where we import our OS or programming language. The next line, starting with ENV, sets our environment variable ENV to APP_HOME / app . This mimics the structure of our project directories, letting Docker know where our app is.

每個Dockerfile的第一行都以FROM.開頭FROM. 這是我們導入操作系統或編程語言的地方。 從ENV開始的下一行將環境變量ENV設置為APP_HOME / app 。 這模仿了項目目錄的結構，讓Docker知道我們的應用程序在哪里。

These lines are part of the Python cloud platform structure and you can read more about them in Google’s cloud documentation.

這些行是Python云平臺結構的一部分，您可以在Google的云文檔中了解有關它們的更多信息。

The WORKDIR line sets our working directory to $APP_HOME. Then, the Copy line makes local files available in the docker container.

WORKDIR行將我們的工作目錄設置為$APP_HOME 。 然后，復制行使本地文件在Docker容器中可用。

The next two lines involve setting up the environment and executing it on the server. The RUN command can be followed with any bash code you would like executed. We use RUN to pip install our requirements. Then CMD to run our HTTP server gunicorn. The arguments in this last line bind our container to$PORT, assign the port a worker, specify the number of threads to use at that port and state the path to the app asapp.main:app.

接下來的兩行涉及設置環境并在服務器上執行環境。 您可以在RUN命令后跟隨您要執行的任何bash代碼。 我們使用RUN點子安裝我們的要求。 然后CMD運行我們的HTTP服務器gunicorn。 最后一行中的參數將我們的容器綁定到$PORT ， $PORT分配一個工作線程，指定在該端口上使用的線程數，并將應用程序的路徑聲明為app.main:app 。

You can add a .dockerignore file to exclude files from your container image. The .dockerignore is used to keep files out of your container. For example, you likely do not want to include your test suite in your container.

您可以添加.dockerignore文件以從容器映像中排除文件。 .dockerignore用于將文件保留在容器之外。 例如，您可能不想在容器中包含測試套件。

To exclude files from being uploaded to Cloud Build, add a.gcloudignore file. Since Cloud Build copies your files to the cloud, you may want to omit images or data to cut down on storage costs.

要排除文件無法上傳到Cloud Build，請添加.gcloudignore文件。 由于Cloud Build將文件復制到云中，因此您可能希望省略圖像或數據以降低存儲成本。

If you would like to use these, be sure to check out the documentation for .dockerignore and .gcloudignorefiles, however, know that the pattern is the same as a.gitignore !

如果您想使用這些文件，請務必查看.dockerignore和.gcloudignore文件的文檔，但是，請知道該模式與.gitignore相同！

在本地構建和啟動Docker容器 (Building and Starting the Docker Container Locally)

Name and build the container with this line. We are calling our container spam-detector.

用此行命名并構建容器。 我們稱我們的容器為spam-detector 。

docker build . -t spam-detector

To start our container we must use this line to specify what ports the container will use. We set the internal port to 8000 and the external port to 5000. We also set the environment variable PORT to 8000 and enter the container name.

要啟動我們的容器，我們必須使用此行來指定容器將使用的端口。 我們將內部端口設置為8000，將外部端口設置為5000。我們還將環境變量PORT設置為8000，然后輸入容器名稱。

PORT=8000 && docker run -p 5000:${PORT} -e PORT=${PORT} spam-detector

Now our app should be up and running in our local Docker container.

現在，我們的應用程序應該已經在本地Docker容器中啟動并運行了。

Let’s send some JSONs to the app at the localhost address provided in the terminal where you’ve run the build.

讓我們通過運行構建的終端中提供的localhost地址向應用發送一些JSON。

使用Postman測試應用 (Testing the app with Postman)

Postman is a software development tool that enables people to test calls to APIs. Postman users enter data. The data is sent to a web server address. Information is returned as a response or an error, which Postman presents to the user.

Postman是一種軟件開發工具，使人們可以測試對API的調用。 郵遞員用戶輸入數據。 數據被發送到Web服務器地址。 信息作為響應或錯誤返回，郵遞員將其呈現給用戶。

Postman makes it easy to test our route. Open up the GUI and

郵遞員可以輕松測試我們的路線。 打開GUI，然后

• Select POST and paste the URL, adding the route as needed
  選擇POST并粘貼URL，根據需要添加路由
• Click Body and then raw
  單擊主體，然后單擊原始
• Select JSON from the dropdown to the right
  從右側的下拉列表中選擇JSON

Be sure to use “text” as the key in your JSON, or the app will throw an error. Place any text you would like the model to process as the value. Now hit send!

確保在JSON中使用“文本”作為鍵，否則應用程序將引發錯誤。 將您希望模型處理的任何文本作為值。 現在點擊發送！

Then view the result in Postman! It looks like our email was categorized as ham. If you receive an error be sure you’ve used the correct key and have the route extension /predict in the POST URL.

然后在郵遞員中查看結果！ 看來我們的電子郵件被歸類為火腿。 如果收到錯誤，請確保您使用了正確的密鑰，并且在POST URL中具有路由擴展名/predict 。

Let’s try an email from my Gmail spam folder.

讓我們嘗試從我的Gmail垃圾郵件文件夾發送一封電子郵件。

Hmm, it looks like we are running a different model than Google.

嗯，看來我們運行的模型與Google不同。

Now let’s test the app without Postman using just the command line.

現在，讓我們僅使用命令行在沒有Postman的情況下測試應用程序。

使用curl測試應用 (Testing the app with curl)

Curl can be a simple tool for testing that allows us to remain in a CLI. I had to do some tweaking to get the command to work with the app, but adding the flags below resolved the errors.

Curl可以是一個簡單的測試工具，可以讓我們保留在CLI中。 我必須進行一些調整才能使命令與應用程序一起使用，但是在下面添加標志可以解決錯誤。

Open the terminal and insert the following. Change the text value to see what the model classifies as spam and ham.

打開終端并插入以下內容。 更改文本值以查看模型歸類為垃圾郵件和火腿的內容。

curl -H "Content-Type: application/json" --request POST -d '{"text": "Spam is my name now give all your money to me"}' http://127.0.0.1:5000/predict

The result, or an error, will populate in the terminal!

結果或錯誤將在終端中填充！

{"result":"ham"}

Now let’s get the app deployed to Google Cloud Platform so anyone can use it.

現在，讓我們將應用程序部署到Google Cloud Platform，以便任何人都可以使用它。

Docker Images和Google Cloud Registry (Docker Images and Google Cloud Registry)

GCP Cloud Build allows you to build containers remotely using the instructions contained in Dockerfiles. Remote builds are easy to integrate into CI/CD pipelines. They also save local computational time and energy as Docker uses lots of RAM.

GCP Cloud Build允許您使用Dockerfiles中包含的說明遠程構建容器。 遠程構建易于集成到CI / CD管道中。 由于Docker使用大量RAM，它們還節省了本地計算時間和精力。

Once we have our Dockerfile ready, we can build our container image using Cloud Build.

一旦我們準備好Dockerfile，就可以使用Cloud Build構建我們的容器映像。

Run the following command from the directory containing the Dockerfile:

從包含Dockerfile的目錄中運行以下命令：

gcloud builds submit --tag gcr.io/PROJECT-ID/container-name

Note: Replace PROJECT-ID with your GCP project ID and container-name with your container name. You can view your project ID by running the command gcloud config get-value project.

注意：將PROJECT-ID替換為GCP項目ID，并將container-name替換為容器名稱。 您可以通過運行命令gcloud config get-value project來查看您的項目ID。

This Docker image now accessible at the GCP container registry or GCR and can be accessed via URL with Cloud Run.

現在，可以在GCP容器注冊表或GCR上訪問此Docker映像，并且可以通過Cloud Run通過URL訪問。

使用CLI部署容器映像 (Deploy the container image using the CLI)

1. Deploy using the following command:
   使用以下命令進行部署：

gcloud run deploy --image gcr.io/PROJECT-ID/container-name --platform managed

Note: Replace PROJECT-ID with your GCP project ID and container-name with your containers’ name. You can view your project ID by running the command gcloud config get-value project.

注意：將PROJECT-ID替換為GCP項目ID，并將container-name替換為容器的名稱。 您可以通過運行命令gcloud config get-value project來查看您的項目ID。

2. You will be prompted for service name and region: select the service name and region of your choice.

2.系統將提示您輸入服務名稱和區域：選擇所需的服務名稱和區域。

3. You will be prompted to allow unauthenticated invocations: respond y if you want public access, and n to limit IP access to resources in the same google project.

3.系統將提示您允許未經授權的調用 ：響應y如果你想公共訪問， n限制IP訪問的資源在同谷歌項目。

4. Wait a few moments until the deployment is complete. On success, the command line displays the service URL.

4.等待片刻，直到完成部署。 成功后，命令行將顯示服務URL。

5. Visit your deployed container by opening the service URL in a web browser.

5.通過在Web瀏覽器中打開服務URL，訪問已部署的容器。

使用GUI部署容器映像 (Deploy the container image using the GUI)

Now that we have a container image stored in GCR, we are ready to deploy our application. Visit GCP cloud run and click create service, be sure to set up billing as required.

現在我們已經在GCR中存儲了一個容器映像，現在可以部署我們的應用程序了。 訪問GCP云運行并點擊創建服務，請確保根據需要設置結算信息。

Select the region you would like to serve and specify a unique service name. Then choose between public or private access to your application by choosing unauthenticated or authenticated, respectively.

選擇您要提供服務的區域并指定唯一的服務名稱。 然后，分別通過選擇未認證或已認證來選擇對應用程序的公共或私人訪問。

Now we use our GCR container image URL from above. Paste in the URL or click select and find it using a dropdown list. Check out the advanced settings to specify server hardware, container port and additional commands, maximum requests and scaling behaviors.

現在，我們從上方使用GCR容器圖片網址。 粘貼在URL中，或單擊“選擇”并使用下拉列表找到它。 檢查高級設置以指定服務器硬件，容器端口和其他命令，最大請求數和擴展行為。

Click create when you’re ready to build and deploy!

準備好構建和部署時，請單擊創建！

You’ll be brought to the GCP Cloud Run service details page where you can manage the service and view metrics and build logs.

您將被帶到GCP Cloud Run服務詳細信息頁面，您可以在其中管理服務并查看指標和構建日志。

Click the URL to view your deployed application!

單擊URL查看已部署的應用程序！

Congratulations! You have just deployed an application packaged in a container to Cloud Run.

恭喜你！ 您剛剛將打包在容器中的應用程序部署到了Cloud Run。

You only pay for the CPU, memory, and networking consumed during request handling. That being said, be sure to shut down your services when you do not want to pay for them!

您只需為請求處理期間消耗的CPU，內存和網絡付費。 話雖如此，當您不想為服務付費時，請務必將其關閉！

結論 (Conclusion)

We’ve covered setting up an app to serve a model and building docker containers locally. Then we dockerized our app and tested it locally. Next, we stored our docker image in the cloud and used it to build an app on Google Cloud Run.

我們已經介紹了設置應用程序以提供模型并在本地構建Docker容器的內容。 然后，我們對應用程序進行了docker化并在本地進行了測試。 接下來，我們將docker映像存儲在云中，并使用它在Google Cloud Run上構建應用程序。

Getting any decently good model out quickly can have significant business and tech value. Value from having something people can immediately use and from having software deployed that a data scientist can tune later.

快速推出任何體面的好的模型可以具有巨大的業務和技術價值。 擁有人們可以立即使用的東西以及部署數據科學家可以稍后進行調整的軟件的價值。

We hope this content is informative and helpful, let us know what you are looking to learn more about in the software, development and machine learning space!

我們希望該內容能為您提供有用的信息，讓我們知道您想在軟件，開發和機器學習領域中進一步學習的內容！