graphpaper | semi/signal

Posts Tagged ‘graphpaper’

Encapsulating Web Workers

Jan 17 2020 · Web Technologies

Constructing Web Workers

There’s generally 2 ways to construct a Web Worker…

Passing a URL to the Javascript file:

const myWorker = new Worker('worker.js');

Or, creating a URL with the Javascript code (as a string). This is done by creating a Blob from the string and passing the Blob to URL.createObjectURL:

const myWorker = new Worker(
    URL.createObjectURL(new Blob([...], {type: 'application/javascript'}))
);

In Practice

With GraphPaper, I’ve used the former approach for the longest while, depending on the caller to construct and inject the worker into GraphPaper.Canvas:

const canvas = new GraphPaper.Canvas(
    document.getElementById('paper'),                        // div to use
    window,                                                  // parent window 
    new Worker('../dist/connector-routing-worker.min.js')    // required worker for connector routing
);

This technically works but, in practice, there’s 2 issues here:

There’s usually a few hoops to go through for the caller to actually get the worker Javascript file in a location that is accessible by the web server. This could mean manually moving the file, additional configuration, additional tooling, etc.
GraphPaper.Canvas is responsible for dealing with whether a worker is used or not, which worker, how many workers are used, etc. These aren’t concerns that should bubble up to the caller. You could make a case that caller should have the flexibility to swap in a worker of their choice (a strategy pattern), that’s a fair point, but I’d argue that the strategy here is what the worker is executing not the worker itself and I haven’t figured out a good interface for what that looks like.

So, I worked to figure out how to construct the worker within GraphPaper.Canvas using URL.createObjectURL(), and this is where things got trickier. The GraphPaper codebase is ES6 and uses ES6 modules, I use rollup with babel to produce distribution files the primary ones being minified IIFE bundles (IIFE because browser support for ES6 modules is still very much lacking). One of these bundles is the code for the worker (dist/connector-routing-worker.js), which I’d need to:

Encapsulate it into a string that can be referenced within the source
Create a Blob from the string
Create a URL from the Blob using URL.createObjectURL()
Pass the URL to the Worker constructor, new Worker(url)

The latter steps are straightforward function calls, but the first is not clear cut.

Repackaging with Rollup

After producing the “distribution” code for the worker, what I needed was to encapsulate it into a string like this (the “worker-string-wrap”):

const workerStringWrap = `
    const ConnectorRoutingWorkerJsString = \`
        ${workerCode}
    \`; 
    export { ConnectorRoutingWorkerJsString }`
;

Writing that out to a file, I could then easily import it as just another ES6 module (and use the string to create a URL for the worker), then build and produce the distribution file for GraphPaper.

I first tried doing this with a nodejs script, but creating a rollup plugin proved a more elegant solution. Rollup plugins are aren’t too difficult to create but I did find the documentation a bit convoluted. Simply, rollup will execute certain functions (hooks) at appropriate points during the build process. The hook needed in this scenario is writeBundle, which can be used to get the code of the produced bundle and do something with it (in this case, write it out to a file).

// rollup-plugin-stringify-worker.js

const fs = require('fs');

const stringifyWorkerPlugin = function (options) {
    return {
        name: 'stringifyWorkerPlugin',
        writeBundle(bundle) {
            console.log(`Creating stringified worker...`);

            // Note: options.srcBundleName and options.dest are expected args from the rollup config

            const workerCode = bundle[options.srcBundleName].code;
            const workerStringWrap = `const ConnectorRoutingWorkerJsString = \`${workerCode}\`; export { ConnectorRoutingWorkerJsString }`;
            fs.writeFile(options.dest, workerStringWrap, function(err) {
                // ...
            });
        }
    };
};


export default stringifyWorkerPlugin;

The plugin is setup within a rollup config file:

import stringifyWorker from './build/rollup-plugin-stringify-worker';

// ...

{
  input: 'src/Workers/ConnectorRoutingWorker.js',
  output: {
    format: 'iife',
    file: 'dist/workers/connector-routing-worker.min.js',
    name: 'ConnectorRoutingWorker',
    sourcemap: false,
  },
  plugins: [
    babel(babelConfig),
    stringifyWorker(
      {
        "srcBundleName": "connector-routing-worker.min.js",
        "dest": "src/Workers/ConnectorRoutingWorker.string.js"
      }
    )
  ],        
},

// ...

Note that addtional config blocks for components that use ConnectorRoutingWorker.string.js (e.g. the GraphPaper distribution files), need to be placed after the block shown above.

The overall process looks like this:

Creating the Worker

The worker can now be created within the codebase as follows:

import {ConnectorRoutingWorkerJsString} from './Workers/ConnectorRoutingWorker.string';

// ...

const workerUrl = URL.createObjectURL(new Blob([ ConnectorRoutingWorkerJsString ]));
const connectorRoutingWorker = new Worker(workerUrl);

// ...

The Future

Looking ahead, I don’t really see a good solution here. Better support for ES6 modules in the browser would be a step in the right direction, but what is really needed is a way to declare a web worker as a module and the ability to import and construct a Worker with that module.