Browsers are currently shipping SharedArrayBuffer gated behind Cross Origin Opener Policy (COOP) and Cross Origin Embedder Policy (COEP) headers. Pthreads code will not work in deployed environment unless these headers are correctly set. For more information click this
Emscripten has support for multithreading using SharedArrayBuffer in browsers. That API allows sharing memory between the main thread and web workers as well as atomic operations for synchronization, which enables Emscripten to implement support for the Pthreads (POSIX threads) API. This support is considered stable in Emscripten.
As of Sep 2019, some browsers have disabled SharedArrayBuffer due to the Spectre set of vulnerabilities. Until it is restored you can still experiment with it if you flip a pref in those browsers. In other browsers (like Chrome on desktop), SharedArrayBuffer is fully enabled by default and you don’t need to flip any flags.
By default, support for pthreads is not enabled. To enable code generation for pthreads, the following command line flags exist:
-pthreadwhen compiling any .c/.cpp files, AND when linking to generate the final output .js file.
-s PTHREAD_POOL_SIZE=<integer>to specify a predefined pool of web workers to populate at page preRun time before application main() is called. This is important because if the workers do not already exist then we may need to wait for the next browser event iteration for certain things, see below.
There should be no other changes required. In C/C++ code, the preprocessor check
#ifdef __EMSCRIPTEN_PTHREADS__ can be used to detect whether Emscripten is currently targeting pthreads.
It is not possible to build one binary that would be able to leverage multithreading when available and fall back to single threaded when not. The best you can do is two separate builds, one with and one without threads, and pick between them at runtime.
-s PROXY_TO_PTHREAD: In this mode your original
main()is replaced by a new one that creates a pthread and runs the original
main()on it. As a result, your application’s
main()is run off the browser main (UI) thread, which is good for responsiveness. The browser main thread does still run code when things are proxied to it, for example to handle events, rendering, etc. The main thread also does things like create pthreads for you, so that you can depend on them synchronously.
Note that Emscripten has the
--proxy-to-worker linker flag which sounds similar
but is unrelated. That flag does not use pthreads or SharedArrayBuffer, and
instead uses a plain Web Worker to run your main program (and postMessage to
proxy messages back and forth).
The Web allows certain operations to only happen from the main browser thread,
like interacting with the DOM. As a result, various operations are proxied to
the main browser thread if they are called on a background thread. See
bug 3495 for
more information and how to try to work around this until then. To check which
operations are proxied, you can look for the function’s implementation in
the JS library (
src/library_*) and see if it is annotated with
__proxy: 'sync' or
__proxy: 'async'; however, note that the browser
itself proxies certain things (like some GL operations), so there is no
general way to be safe here (aside from not blocking on the main browser
In addition, Emscripten currently has a simple model of file I/O only happening on the main application thread (as we support JS plugin filesystems, which cannot share memory); this is another set of operations that are proxied.
Proxying can cause problems in certain cases, see the section on blocking below.
The Web API for atomics does not allow blocking on the main thread
Atomics.wait doesn’t work there). Such blocking is
necessary in APIs like
pthread_join and anything that uses a futex wait
under the hood, like
pthread_mutex_lock(). To make them work, we use a busy-wait on the main
browser thread, which can make the browser tab unresponsive, and also wastes
power. (On a pthread, this isn’t a problem as it runs in a Web Worker, where
we don’t need to busy-wait.)
Busy-waiting on the main browser thread in general will work despite the
downsides just mentioned, for things like waiting on a lightly-contended mutex.
However, things like
are often intended to block for long periods of time, and if that
happens on the main browser thread, and while other threads expect it to
respond, it can cause a surprising deadlock. That can happen because of
proxying, see the previous section. If the main thread blocks while a worker
attempts to proxy to it, a deadlock can occur.
The bottom line is that on the Web it is bad for the main browser thread to
wait on anything else. Therefore by default Emscripten warns if
pthread_cond_wait happen on the main browser thread,
and will throw an error if
ALLOW_BLOCKING_ON_MAIN_THREAD is zero
(whose message will point to here).
To avoid these problems, you can use
PROXY_TO_PTHREAD, which as
mentioned earlier moves your
main() function to a pthread, which leaves
the main browser thread to focus only on receiving proxied events. This is
recommended in general, but may take some porting work, if the application
main() was on the main browser thread.
Another option is to replace blocking calls with nonblocking ones. For example
you can replace
pthread_tryjoin_np. This may require
your application to be refactored to use asynchronous events, perhaps through
emscripten_set_main_loop() or Asyncify.
The Emscripten implementation for the pthreads API should follow the POSIX standard closely, but some behavioral differences do exist:
pthread_create()is called, if we need to create a new Web Worker, then that requires returning the main event loop. That is, you cannot call
pthread_createand then keep running code synchronously that expects the worker to start running - it will only run after you return to the event loop. This is a violation of POSIX behavior and will break common code which creates a thread and immediately joins it or otherwise synchronously waits to observe an effect such as a memory write. There are several solutions to this:
emscripten_set_main_loop, or Asyncify).
-s PTHREAD_POOL_SIZE=<integer>. Using a pool creates the Web Workers before main is called, so they can just be used when
-s PROXY_TO_PTHREAD, which will run
main()on a worker for you. When doing so,
pthread_createis proxied to the main browser thread, where it can return to the main event loop as needed.
ALLOW_MEMORY_GROWTH) is especially tricky, see wasm design issue #1271. This currently causes JS accessing the wasm memory to be slow - but this will likely only be noticeable if the JS does large amounts of memory reads and writes (wasm runs at full speed, so moving work over can fix this). This also requires that your JS be aware that the HEAP* views may need to be updated - JS code embedded with
--js-libraryetc will automatically be transformed to use the
GROWABLE_HEAP_*helper functions where
HEAP*are used, but external code that uses
Module.HEAP*directly may encounter problems with views being smaller than memory.
NAME.worker.js is generated alongside the output .js file (where
NAME is the basename of the main file being emitted). That file must be deployed with the rest of the generated code files. By default,
NAME.worker.js will be loaded relative to the main HTML page URL. If it is desirable to load the file from a different location e.g. in a CDN environment, then one can define the
Module.locateFile(filename) function in the main HTML
Module object to return the URL of the target location of the
NAME.worker.js entry point. If this function is not defined in
Module, then the default location relative to the main HTML file is used.
Any code that is compiled with pthreads support enabled will currently only work in the Firefox Nightly channel, since the SharedArrayBuffer specification is still in an experimental research stage before standardization. There exists two test suites that can be used to verify the behavior of the pthreads API implementation in Emscripten:
Please check these first in case of any issues. Bugs can be reported to the Emscripten bug tracker as usual.