By: Content by Edward Krueger and Josh Farmer, and Douglas Franklin.

作者： 愛德華·克魯格 ( Edward Krueger) 和 喬什·法默 ( Josh Farmer )以及 道格拉斯·富蘭克林 ( Douglas Franklin)的內容 。

When building an application that performs time-consuming, complex, or resource-intensive tasks, it can be frustrating to wait for these to complete within the front end application. Additionally, complex tasks in the front end can time-out. Redis Queue fixes this by pushing more sophisticated tasks to a worker for processing.

在構建執行耗時，復雜或資源密集型任務的應用程序時，等待這些任務在前端應用程序中完成可能會令人沮喪。 此外，前端的復雜任務可能會超時。 Redis Queue通過將更復雜的任務推送給工作人員進行處理來解決此問題。

Using Redis with Redis Queue allows you to enter those complex tasks into a queue, so the Redis worker executes these tasks outside of your application’s HTTP server.

通過將Redis與Redis Queue一起使用，您可以將那些復雜的任務輸入隊列，因此Redis worker在應用程序的HTTP服務器之外執行這些任務。

In this article, we will build an app that enqueues jobs with Redis queue, performs a function on those jobs and returns the result of the function.

在本文中，我們將構建一個應用程序，該應用程序使用Redis隊列使作業排隊，對這些作業執行功能，并返回該功能的結果。

Here is the link to the Github Repository with our code for this project.

這是Github信息庫的鏈接，以及該項目的代碼。

什么是Redis？ (What is Redis?)

Redis is an open-source, in-memory database, cache, and message broker. Messages handled by Redis are essentially JSONs. Redis is ideal for quickly and easily working with specific data types in a temporary database, and provides very rapid access and delivery of queries.

Redis是一個開源的內存數據庫，緩存和消息代理。 Redis處理的消息本質上是JSON。 Redis是快速，輕松地使用臨時數據庫中特定數據類型的理想選擇，并且可以非常快速地訪問和交付查詢。

For us, Redis offers two benefits. First, it pushes complex tasks to another space to be processed. Second, it is easier for the developer to handle complex actions by splitting the task into separate functions, the main application and the queue.

對于我們來說，Redis提供了兩個好處。 首先，它將復雜的任務推到另一個要處理的空間。 其次，通過將任務分為獨立的功能，主應用程序和隊列，開發人員可以更輕松地處理復雜的操作。

For this application, we’ll be using Redis to hold our queue of JSON messages. Redis can be a stand-alone database, accessible by many computers or systems. In our example, we will be using it as a local memory store to support our application.

對于此應用程序，我們將使用Redis來保存JSON消息隊列。 Redis可以是一個獨立的數據庫，可以被許多計算機或系統訪問。 在我們的示例中，我們將使用它作為本地內存存儲來支持我們的應用程序。

什么是Redis Queue？ (What is Redis Queue?)

Redis Queue is a python library for queueing jobs for background processing. Since many hosting services will time out on long HTTP requests, it is best to design APIs to close requests as quickly as possible. Redis Queue allows us to do this by pushing tasks to a queue and then to a worker for processing.

Redis Queue是一個用于對作業進行排隊以進行后臺處理的python庫。 由于許多托管服務會在較長的HTTP請求上超時，因此最好設計API以盡快關閉請求。 Redis Queue允許我們通過將任務推入隊列，然后推給工作人員進行處理來做到這一點。

Using Redis in conjunction with Redis Queue allows you to request input from the user, return a validation response to the user, and queue up processes in the background. All without the front end user having to wait for those processes to complete. Processes could be anything from Machine Learning models, to duplicating an image to complex simulations.

將Redis與Redis Queue結合使用可以使您從用戶請求輸入，向用戶返回驗證響應，并在后臺排隊進程。 所有這些都無需前端用戶等待這些過程完成。 從機器學習模型到將圖像復制到復雜的模擬，過程可以是任何東西。

Anything that takes longer to complete than you would like to have taken place on the front end of your application belongs in the Redis Queue.

Redis隊列中需要完成的時間比在應用程序前端所需的時間長。

模塊化應用程序結構 (Modular App Structure)

We will cover using Docker to run the Redis database and initialize the Redis Queue worker. The worker will allow us to process the jobs in our application’s queue.

我們將介紹如何使用Docker運行Redis數據庫并初始化Redis Queue worker。 工作者將允許我們處理應用程序隊列中的作業。

We’ll go over each of these files in more detail in the following sections.

在以下各節中，我們將詳細介紹每個文件。

We’ll discuss the following constituent files of our app, main.py, functions.py, and redis_resc.py.

我們將討論應用程序的以下組成文件： main.py ， functions.py和redis_resc.py 。

There are two ways to initiate the docker container. We will be covering the standard process one step at a time in this article. However, the docker container can be created using docker-compose, which we will cover in a separate article.

有兩種啟動docker容器的方法。 本文將一次一步地介紹標準過程。 但是，可以使用docker-compose創建docker容器，我們將在另一篇文章中介紹。

安裝Docker和Redis (Installing Docker and Redis)

Docker is a containerization service. This means Docker runs code within a container that has within it the dependencies required for the application to run reliably from one system to another. It ensures our applications run consistently, from development to testing to production.

Docker是一種容器化服務。 這意味著Docker在容器中運行代碼，該容器中包含應用程序從一個系統可靠地運行到另一個系統所需的依賴關系。 它確保我們的應用程序從開發到測試再到生產，始終如一地運行。

Follow this link to the Docker documentation for a comprehensive installation guide to help you select the correct version of Docker for your operating system.

單擊此鏈接以獲取完整的安裝指南的Docker文檔 ，以幫助您為您的操作系統選擇正確的Docker版本。

Once you have Docker installed on your system, you will need to install Redis.

一旦在系統上安裝了Docker，就需要安裝Redis。

For Mac users, the home-brew command brew install redis will work. Windows requires a download from Github. With Linux, you can download the latest tarball directly from Redis.

對于Mac用戶，home-brew命令brew install redis將起作用。 Windows需要從Github下載 。 使用Linux，您可以直接從Redis下載最新的tarball。

自述概述 (Readme Overview)

To run our application, we must understand the instructions on the Readme file.

要運行我們的應用程序，我們必須了解自述文件上的說明 。

We will be using Pipenv to create a virtual environment and install the appropriate packages. Install the packages for this program with the command pipenv install -- dev. Adding the --dev flag installs development packages and production requirements. Before moving to the next step, activate the environment with the command pipenv shell.

我們將使用Pipenv創建虛擬環境并安裝適當的軟件包。 使用命令pipenv install -- dev該程序的軟件包。 添加--dev標志將安裝開發包和生產要求。 進行下一步之前，請使用命令pipenv shell激活環境。

We will have to start three services to have the app function correctly: Redis Database, a Redis Queue worker, and the Flask application. Two of these services will begin within the same terminal as the one we used to install our virtual environment and enter the shell.

我們將必須啟動三個服務才能使應用程序正常運行：Redis數據庫，Redis隊列工作器和Flask應用程序。 這些服務中的兩項將在與我們用于安裝虛擬環境并進入Shell的終端相同的終端中啟動。

First, run the command:

首先，運行命令：

docker pull redis

This will pull the Redis image from Docker Hub.

這將從Docker Hub中提取Redis映像。

Then, run the command:

然后，運行命令：

docker run -d -p 6379:6379 redis

This command serves several purposes. First, it initializes the docker container. The -d flag runs the Redis container in the background, freeing up your terminal for the next step. The -p flag publishes your container’s port or ports to the host. Lastly, 6379:6379 binds port 6379 inside the docker container to port 6379 on the localhost. Without binding the port, you will run into issues when the local machine and processes within your docker container attempt to communicate with one another.

此命令有幾個用途。 首先，它初始化docker容器。 -d標志在后臺運行Redis容器，從而釋放您的終端以進行下一步。 -p標志將容器的一個或多個端口發布到主機。 最后， 6379:6379將docker容器內的端口6379:6379綁定到本地主機上的端口6379。 如果不綁定端口，當docker容器中的本地計算機和進程嘗試相互通信時，就會遇到問題。

The final step before running our application is starting the Redis Queue worker. Since we are running the Redis image in detached mode, we can do this step within the same terminal.

運行我們的應用程序之前的最后一步是啟動Redis Queue worker。 由于我們以分離模式運行Redis映像，因此我們可以在同一終端中執行此步驟。

Start the Redis worker with the command rq worker . The rq command is only available to the terminal if the Redis Queue package is installed, we installed the dependencies with Pipenv install, and entered the environment with pipenv shell. Running the rq command will allow Redis Queue to begin listening for jobs to enter into the queue and start processing these jobs.

使用命令rq worker啟動Redis rq worker 。 僅當安裝了Redis Queue軟件包，使用Pipenv install安裝依賴項并使用pipenv shell進入環境時， rq命令才對終端可用。 運行rq命令將使Redis Queue開始偵聽要進入隊列的作業并開始處理這些作業。

初始化Flask應用 (Initializing the Flask App)

We are now ready to initialize the flask app. On a development server, this is done with the command:

現在我們準備初始化flask應用程序。 在開發服務器上，這是通過以下命令完成的：

export FLASK_APP=app.main:app && flask run — reload

The command gunicorn app.main:app will run this on a production server. Gunicorn requires a Unix platform to run. Therefore, this command will work if you are using macOS or a Linux based system, but not for windows.

命令gunicorn app.main:app將在生產服務器上運行此命令。 Gunicorn需要Unix平臺才能運行。 因此，如果您使用的是macOS或基于Linux的系統，則此命令將起作用，但不適用于Windows。

Now that the services are running, let’s move to the app files.

現在，這些服務正在運行，讓我們移至應用程序文件。

Flask應用程序組件 (Flask App Components)

redis_resc.py (redis_resc.py)

This file sets up the Redis connection and the Redis Queue.

此文件設置Redis連接和Redis隊列。

"""Sets up the redis connection and the redis queue."""
import osimport redis
from rq import Queueredis_conn = redis.Redis(host=os.getenv("REDIS_HOST", "127.0.0.1"),port=os.getenv("REDIS_PORT", "6379"),password=os.getenv("REDIS_PASSWORD", ""),
)redis_queue = Queue(connection=redis_conn)

The variable redis_conn defines the connection parameters for this connection, and redis_queue uses the Queue method. The Queue method initiates the queue, and any name can be given. Common naming patters are ‘low,’ ‘medium’ and ‘high.’ By providing the Queue method no args, we are instructing it to use the default queue. Aside from the name, we are simply passing our method the connection string to the Redis store.

變量redis_conn定義了此連接的連接參數，并且redis_queue使用Queue方法。 Queue方法啟動隊列，并且可以指定任何名稱。 常見的命名方式是“低”，“中”和“高”。 通過不為Queue方法提供args，我們指示它使用默認隊列。 除了名稱之外，我們只是將連接字符串傳遞給Redis存儲。

functions.py (functions.py)

Functions.py is where we define the functions to use in the Redis queue.

Functions.py是我們定義要在Redis隊列中使用的函數的位置。

What some_long_function does is up to you. It can be anything you want with done with the data passed to the worker. This could be anything from machine learning, to image manipulation, to using the information to process queries in a connected database. Anything you want the backend to do so the front end doesn't wait would be placed within this function.

some_long_function功能取決于您。 完成傳遞給工作程序的數據后，您可以執行任何操作。 從機器學習到圖像處理，再到使用信息來處理連接的數據庫中的查詢，都可以是任何東西。 您希望后端執行任何使前端不等待的操作都將放置在此函數中。

For our example, this function serves a couple of purposes. First, the job variable is created to retrieve the current job being worked. We use time.sleep(10) to demonstrate the different job status codes as the job is being processed. We will go into more detail on the job status codes later in the article. Finally, we return a JSON that contains information about the job and the results of processing the job.

在我們的示例中，此功能有兩個目的。 首先，創建作業變量以檢索當前正在工作的作業。 在處理作業時，我們使用time.sleep(10)演示不同的作業狀態代碼。 我們將在本文后面詳細介紹作業狀態代碼。 最后，我們返回一個JSON，其中包含有關作業和處理作業的結果的信息。

主程序 (Main.py)

Main.py is the primary flask app that drives the application. More details on setting up a flask app are outlined in this medium article.

Main.py是驅動該應用程序的主要燒瓶應用程序。 這篇中篇文章概述了有關設置flask應用程序的更多詳細信息。

The first route function, resource_not_found , is designed to handle any 404 errors that result from incorrect requests to the application. The route function defined as home is the default route that runs when you navigate to the home route. This route runs when you perform a GET request to http://127.0.0.1:8000 or the localhost URL. This route designed to let you know the flask app is running.

第一個路由函數resource_not_found旨在處理由于對應用程序的不正確請求而導致的任何404錯誤。 定義為home的路由功能是導航到本地路由時運行的默認路由。 當您執行對http://127.0.0.1:8000或本地主機URL的GET請求時，此路由將運行。 此路由旨在讓您知道flask應用程序正在運行。

We can test it with Postman as seen below.

我們可以使用郵遞員對其進行測試，如下所示。

The remaining routes, specific for the Redis Queue, will be covered in more detail below.

其余的路由(特定于Redis隊列)將在下面詳細介紹。

/入隊 (/enqueue)

The /enqueue route creates the task and enters the task into the queue. Redis Queue accepts things such as key-value pairs so that you can create a post request with a dictionary, for example:

/enqueue路由創建任務并將任務輸入隊列。 Redis Queue接受鍵值對之類的東西，以便您可以使用字典創建發布請求，例如：

{“hello”: “world”}

Submitting a JSON post request to the /enqueue route will tell the application to enqueue the JSON to the function some_long_function within the function.py file for processing.

向/enqueue路由提交JSON發布請求將告訴應用程序將JSON排隊到function.py文件中的some_long_function 函數進行處理。

The /enqueue route returns the job_id of the task created once it has done so. Take note of the job_id, as we will use it as part of the URL string for the application’s remaining routes.

/enqueue路由返回創建的任務的job_id 。 請注意job_id ，因為我們會將其用作應用程序其余路由的URL字符串的一部分。

/檢查狀態 (/check_status)

This route takes the job_id and checks its status in the Redis Queue. The full URL of this route is http://127.0.0.1:8000?job_id=JobID. Where JobID is the job_id that was generated when the /enqueue route created the job.

該路由采用job_id并在Redis隊列中檢查其狀態。 該路由的完整URL為http://127.0.0.1:8000?job_id=JobID 。 其中JobID是/enqueue路由創建作業時生成的job_id 。

The /check_status route will return a JSON that contains the job_id passed in the URL and the status of the job using the get_status method of rq.job. Remember within the some_long_function, the time.sleep at the start of the function. If you hit this route with a get request before the ten-second timer, the status will show ‘Queue.’ After some_long_function completes, the status will return as ‘finished.’

/check_status路由將返回一個JSON，其中包含使用URL中傳遞的job_id和使用get_status方法的工作狀態。 還記得內some_long_function的time.sleep在函數的開始。 如果您在十秒計時器之前通過獲取請求觸及此路線，則狀態將顯示“隊列”。 some_long_function完成后，狀態將返回“完成”。

/ get_result (/get_result)

Once a job is shown as ‘finished’ by the /check_status route, get_result will return the results of the processed job. The results are determined by what you have set up within some_long_function. This URL follows the same structure as /check_status so it is:

一旦通過/check_status route將作業顯示為“完成”， get_result將返回已處理作業的結果。 結果取決于您在some_long_function設置的some_long_function 。 該URL與/check_status結構相同，因此為：

http://127.0.0.1:8000?get_result=JobID

Our some_long_function returns data about the job, as seen in the image above.

我們的some_long_function返回有關作業的數據，如上圖所示。

結論 (Conclusion)

In this article, we learned a little about building a queue of tasks with Redis Queue, and how to utilize a Redis database within a docker container. We covered how to build the application that makes it simple to enqueue time-consuming tasks without tying up or hindering our front end performance.

在本文中，我們了解了一些有關使用Redis Queue構建任務隊列的知識，以及如何在Docker容器中利用Redis數據庫。 我們介紹了如何構建應用程序，以使其輕松地完成耗時的任務，而又不會增加或妨礙我們的前端性能。

Here is the link to the Github Repository with our code for this project. Test it out for yourself and try queueing a few JSON tasks and viewing the results!

這是Github信息庫的鏈接，以及該項目的代碼。 自己進行測試，然后嘗試排隊一些JSON任務并查看結果！