webgl 著色器

by Dan Ruta

通過Dan Ruta

在WebAssembly中使用WebGL著色器 (Using WebGL shaders in WebAssembly)

WebAssembly is blazing fast for number crunching, game engines, and many other things, but nothing can quite compare to the extreme parallelization of shaders, running on the GPU.

WebAssembly正在為數字運算，游戲引擎和許多其他事情快速發展，但是沒有什么可以與在GPU上運行的著色器的極端并行化相比。

This is especially so if you’re looking to do some image processing. Usually, on the web, this is done through WebGL, but how would you access its APIs when using WebAssembly?

如果您要進行一些圖像處理，則尤其如此。通常，在Web上，這是通過WebGL完成的，但是使用WebAssembly時如何訪問其API？

配置 (Setting up)

We’ll very briefly go through setting up an example project, then we’ll look at how an image can be loaded as a texture. Then, in a separate context, we’ll apply an edge detection GLSL shader to the image.

我們將簡要地設置一個示例項目，然后看一下如何將圖像作為紋理加載。然后，在單獨的上下文中，我們將對圖像應用邊緣檢測GLSL著色器。

All the code is in a repo here, if you’d prefer to jump straight to that. Note that you have to serve your files via a server for WebAssembly to work.

如果您希望直接跳轉到此處，所有代碼都在此處的存儲庫中。請注意，您必須通過服務器提供文件才能使WebAssembly正常工作。

As a prerequisite, I’m going to assume you already have your WebAssembly project set up. If not, you can check out the article here on how to do it, or just fork the repo linked above.

作為前提，我將假設您已經設置了WebAssembly項目。如果沒有，您可以在此處查看有關操作方法的文章，也可以只分叉上面鏈接的倉庫。

For demoing the below code, I’m using a basic html file which serves only to load an image, get its imageData, and pass it to the WebAssembly code using the ccallArrays function.

為了演示以下代碼，我使用一個基本的html文件，該文件僅用于加載圖像，獲取其imageData并使用ccallArrays函數將其傳遞給WebAssembly代碼。

As for the C++ code, there is an emscripten.cpp file which manages and routes method calls to context instances created in the Context.cpp file. The Context.cpp file is structured as follows:

對于C ++代碼，有一個emscripten.cpp文件，用于管理方法調用并將其路由到Context.cpp文件中創建的上下文實例。 Context.cpp文件的結構如下：

匯編 (Compilation)

WebGL is based on and follows the OpenGL ES (Embedded Systems) spec, which is a subset of OpenGL. When compiling, emscripten will map our code to the WebGL API.

WebGL基于并遵循OpenGL ES(嵌入式系統)規范，該規范是OpenGL的子集。編譯時，emscripten會將我們的代碼映射到WebGL API。

There are a couple of different versions we can target. OpenGL ES 2 maps to WebGL 1, whereas OpenGL ES 3 maps to WebGL 2. By default you should target WebGL 2, as it comes with some free optimizations and improvements.

我們可以定位幾個不同的版本。 OpenGL ES 2映射到WebGL 1，而OpenGL ES 3映射到WebGL2。默認情況下，您應該以WebGL 2為目標，因為它帶有一些免費的優化和改進功能。

To do this, we must add the USE_WEBGL2=1 flag to the compilation.

為此，我們必須在編譯中添加USE_WEBGL2=1標志。

If you are planning to use some OpenGL ES features not present in the WebGL spec, you can use the FULL_ES2=1 and/or FULL_ES3=1 flags.

如果您打算使用WebGL規范中未提供的某些OpenGL ES功能，則可以使用FULL_ES2=1和/或FULL_ES3=1標志。

To be able to handle large textures/images, we can also add the ALLLOW_MEMORY_GROWTH=1 flag. This removes the memory limit of the WebAssembly program, at the cost of some optimizations.

為了能夠處理大的紋理/圖像，我們還可以添加ALLLOW_MEMORY_GROWTH=1標志。這以一些優化為代價，消除了WebAssembly程序的內存限制。

If you know ahead of time how much memory you’ll need, you can instead use the TOTAL_MEMORY=X flag, where X is the memory size.

如果您提前知道需要多少內存，則可以使用TOTAL_MEMORY=X標志，其中X是內存大小。

So we’re going to end up with something like this:

因此，我們將最終得到如下結果：

emcc -o ./dist/appWASM.js ./dev/cpp/emscripten.cpp -O3 -s ALLOW_MEMORY_GROWTH=1 -s USE_WEBGL2=1 -s FULL_ES3=1 -s WASM=1 -s NO_EXIT_RUNTIME=1 -std=c++1z

Finally, we need the following imports, in our code:

最后，我們需要在代碼中進行以下導入：

#include <emscripten.h>#include <string>#include <GLES2/gl2.h>#include <EGL/egl.h>extern "C" {       #include "html5.h" // emscripten module}

實作 (Implementation)

If you have previous experience with WebGL or OpenGL, then this bit may seem familiar.

如果您以前有使用WebGL或OpenGL的經驗，那么這一點可能看起來很熟悉。

When writing OpenGL, the API will not work until you create a context. This is normally done using platform specific APIs. However, the web is not platform bound, and we can instead use an API integrated into OpenGL ES.

在編寫OpenGL時，除非創建上下文，否則該API將無法工作。通常使用平臺特定的API完成此操作。但是，網絡不受平臺限制，因此我們可以使用集成到OpenGL ES中的API。

The majority of the legwork, however, can be more easily implemented using emscripten’s APIs in the html5.h file. The functions we’re interested in are:

但是，使用html5.h文件中的emscripten的API可以更輕松地實現大部分工作。我們感興趣的功能是：

emscripten_webgl_create_context — This will instantiate a context for the given canvas and attributes
emscripten_webgl_create_context —這將為給定的畫布和屬性實例化上下文
emscripten_webgl_destroy_context — This is needed for cleaning up memory when destructing context instances
emscripten_webgl_destroy_context —銷毀上下文實例時清理內存是必需的
emscripten_webgl_make_context_current — This will assign and switch which context WebGL will render to
emscripten_webgl_make_context_current —這將分配并切換WebGL渲染到的上下文

創建上下文 (Create the context)

To start implementing, you have to first create the canvas elements in your JavaScript code. Then, when using the emscripten_webgl_create_context function, you pass the id of the canvas as the first parameter, with any configurations as the second. The emscripten_webgl_make_context_current function is used to set the new context as the one currently in use.

要開始實施，您必須首先在JavaScript代碼中創建canvas元素。然后，在使用emscripten_webgl_create_context函數時，您將畫布的ID作為第一個參數傳遞，將任何配置作為第二個參數傳遞。 emscripten_webgl_make_context_current函數用于將新上下文設置為當前使用的上下文。

Next, the vertex shader (to specify coordinates) and the fragment shader (to calculate the colour at each pixel) are both compiled, and the program is built.

接下來，編譯頂點著色器(用于指定坐標)和片段著色器(以計算每個像素的顏色)，并構建程序。

Finally, the shaders are attached to the program, which is then linked, and validated.

最后，將著色器附加到程序，然后將其鏈接并驗證。

Though that sounds like a lot, the code for this is as follows:

盡管這聽起來很多，但其代碼如下：

The shader compilation is done within the CompileShader helper function which performs the compilation, printing out any errors:

著色器編譯是在CompileShader幫助器函數中完成的，該函數執行編譯，并打印出所有錯誤：

創建著色器 (Create the shader)

The shader code for this example is minimal, and it just maps each pixel to itself, to display the image as a texture:

此示例的著色器代碼很少，它僅將每個像素映射到自身，以將圖像顯示為紋理：

You can access the canvas’ context in JavaScript in addition to the context in the C++ code, but it must be of the same type, ‘webgl2’. While defining multiple context types does nothing when just using JavaScript, if you do it before creating the webgl2 context in WebAssembly, it will throw an error when the code execution gets there.

除了C ++代碼中的上下文，您還可以在JavaScript中訪問canvas的上下文，但是它必須是相同的類型“ webgl2”。盡管僅使用JavaScript定義多個上下文類型并沒有任何作用，但是如果在WebAssembly中創建webgl2上下文之前進行了定義，則在執行代碼時會引發錯誤。

加載紋理 (Loading the texture)

The first thing to do when applying the shader is to call the emscripten_webgl_make_context_currentfunction to make sure that we are still using the correct context, and glUseProgramto make sure we are using the correct program.

應用著色器時要做的第一件事是調用emscripten_webgl_make_context_current函數以確保我們仍在使用正確的上下文，并glUseProgram來確保我們在使用正確的程序。

Next, we get the indices of the GLSL variables (similar to getting a pointer) via theglGetAttribLocationand glGetUniformLocation functions, so we can assign our own values to those locations. The function used to do that depends on the value type.

接下來，我們通過glGetAttribLocation和glGetUniformLocation獲取GLSL變量的索引(類似于獲取指針) 函數，因此我們可以將自己的值分配給這些位置。用于執行此操作的函數取決于值類型。

For example, an integer, such as the texture location needs glUniform1i, whereas a float would need glUniform1f. This is a good resource for seeing which function you need to use.

例如，整數(例如紋理位置)需要glUniform1i ，而浮點數則需要glUniform1f 。這是查看需要使用哪個功能的好資源。

Next, we get the texture object via glGenTextures, assign it as the active texture, and load the imageData buffer. The vertex and indices buffers are then bound, to set the boundaries of the texture to fill the canvas.

接下來，我們通過glGenTextures獲得紋理對象，將其分配為活動紋理，并加載imageData緩沖區。然后綁定頂點和索引緩沖區，以設置紋理的邊界以填充畫布。

Finally, we clear the existing content, define our remaining variables with data, and draw to the canvas.

最后，我們清除現有內容，使用數據定義剩余的變量，然后繪制到畫布上。

使用著色器檢測邊緣 (Detect edges using a shader)

To add another context, where the edge detection is done, we load a different fragment shader (which applies the Sobel filter), and we bind the width and height as extra variables, in the code.

為了添加另一個完成邊緣檢測的上下文，我們在代碼中加載了一個不同的片段著色器(應用了Sobel濾波器)，并將寬度和高度綁定為額外的變量。

To pick between different fragment shaders, for the different contexts, we just add an if-else statement in the constructor, like so:

為了在不同的片段著色器之間進行選擇，針對不同的上下文，我們只需在構造函數中添加if-else語句，如下所示：

And to load the width and height variables, we add the following to the run function:

為了加載width和height變量，我們將以下內容添加到run函數中：

If you run into an error similar toERROR: GL_INVALID_OPERATION : glUniform1i: wrong uniform function for type, then there’s a mismatched assignment function for the given variable.

如果遇到類似于ERROR: GL_INVALID_OPERATION : glUniform1i: wrong uniform function for type ，則給定變量的賦值函數不匹配。

One thing to look out for when sending the imageData, is to use the correct heap, unsigned integer (the Uint8Array typed array). You can learn more about those here, but if you’re using the ccallArray function, set the ‘heapIn’ config to “HEAPU8”, as seen above.

發送imageData時要注意的一件事是使用正確的堆(無符號整數)(Uint8Array類型的數組)。您可以在此處了解更多信息，但是，如果您使用的是ccallArray函數，則將“ heapIn ”配置設置為“ HEAPU8 ”，如上所示。

If the type is not correct, the texture will still load, but you’re going to be seeing strange renderings, like these:

如果類型不正確，紋理仍會加載，但是您將看到奇怪的渲染，如下所示：

結論 (Conclusion)

We’ve gone through a mini “Hello World!”-style project to show how to load textures and apply GLSL shaders to them in WebAssembly. The complete code is hosted on GitHub here, for further reference.

我們已經完成了一個迷你的“ Hello World！”風格的項目，以展示如何加載紋理并將GLSL著色器應用到WebAssembly中。完整的代碼是在GitHub托管在這里，以備將來參考。

For a real project, you may want to add some additional error handling. I omitted it here, for clarity.

對于真實項目，您可能需要添加一些其他錯誤處理。為了清楚起見，我在這里省略了它。

It may also be more efficient (in the above example) to share data such as the imageData texture between contexts. You can read more about this and more here.

(在上述示例中)在上下文之間共享數據(例如imageData紋理)也可能更為有效。您可以在此處有關此內容的信息。

For some further reading, you can check out this link for common mistakes, or you can look through some demo projects in emscripten’s glbook folder, on GitHub.

若要進一步閱讀，可以查看此鏈接以查找常見錯誤，也可以在GitHub上emscripten的glbook文件夾中瀏覽一些演示項目。

To see WebGL being used in a WebAssembly project, you can check out the dev branch on jsNet, a web based deep learning framework, where I’ll be working on moving heavier computations onto shaders, over the next few weeks (support for WebGL compute shaders via OpenGL ES 3.1 can’t come soon enough ? ).

要查看WebGL在WebAssembly項目中的使用情況，您可以在jsNet上檢查一下dev分支，jsNet是一個基于Web的深度學習框架，在接下來的幾周中，我將在其中致力于將較重的計算轉移到著色器上(支持WebGL計算通過OpenGL ES 3.1創建的著色器還不夠快嗎？)。

Update

更新資料

To see what GPU compute using shaders would look like in WebAssembly, you can check out the repo for GPGPU, a small library I’m working on, with both JavaScript and WebAssembly versions.

若要查看在WebAssembly中使用著色器進行GPU計算的外觀，您可以檢出GPGPU (我正在使用的一個小型庫，包含JavaScript和WebAssembly版本)的存儲庫。