Posts Tagged ‘nodejs’

Deployments with git tags + npm publish

Git tags and npm

Deployment workflows can vary a lot, but what I’ve tended to find ideal is to tag releases on GitHub (or whatever platform, as most have some mechanism to handle releases), the tag itself being the version number of whatever is being deployed, and having a deployment pipeline orchestrate and perform whatever steps are necessary to deploy the application, service, library, etc. This flow is well supported with git, well supported on platforms like GitHub, and is dead simple for developers to pick up and work with (in GitHub, this means filling out a form and hitting “Publish release”).

npm doesn’t play nicely with this workflow. With npm version numbers aren’t tied to git tags, or any external mechanism, but instead to the value defined in the project’s package.json file. So, trying to publish a package via tagging requires some additional steps. The typical solutions seem to be:

  • Update the version in package.json first, then create the tag
  • Use some workflow that include npm version patch to have npm handle the update to package.json and creating the git tag
  • Use an additional tool (e.g. standard-version), that tries to abstract away management of version numbers from both package.json and git tag

None of these options are great; versioning responsibility and authority is pulled away from git and, in the process, additional workflow complexity and, in the latter case, additional dependencies are introduced.

Version 0.0.0

In order to publish with npm, keep versioning authority with git, and maintain a simple workflow that doesn’t include additional steps or dependencies, the following has been working well in my projects:

  • In package.json, set the version number to “0.0.0”; this value is never changed within any git branch and, conceptually, it can be viewed as representing the “dev version” of the library. package.json only has a “non-dev version” for code published to our package repository.
  • In the deployment pipeline (triggered by tagging a release), update package.json with the version from the git tag.

    Most CI systems have some way of getting the tag being processed and working with it. For example, in CircleCI, working with tags formatted like vMAJOR.MINOR.PATCH, we can reference the tag, remove the “v” prefix, and set the version in package.json using npm version as follows:

    npm --no-git-tag-version version ${CIRCLE_TAG:1}

    Note that this update to package.json is only done within the checked-out copy of the code used in the pipeline. The change is never committed to the repo nor pushed upstream.
  • Finally, within the deployment pipeline, publish as usual via npm publish


I haven’t run across any major limitations with this workflow. There is some loss of information captured in the git repository, as the version in package.json is fixed at 0.0.0, but I’ve yet to come across that being an issue. I could potentially see issues if you want to allow developers to do deployments locally via npm publish but, in general, I view local deployments as an anti-pattern when done for anything beyond toy projects.

Writing and testing ES modules

The toolchain

ES modules are one of the more exciting additions to the Javascript language. Effectively being able to break-off and modularize has continually led to better code and development practices in my experience. For server-side Javascript, Node and its associated module system took hold, but there was nothing comparable for browsers. However, Node-based toolchains to produce frontend code also became a thing, doing rollup, transpilation, minification, etc. This wasn’t necessarily a bad thing, and came with some noted benefits such as better support for unit testing scaffolds and integration into CI pipelines, but it also set a stage for increasingly complex toolchains and an ecosystem whereby frontend components were Node-based server-side components first, and transformed into frontend components after. The latter, in turn, led to a state where you were always working with a toolchain (it wasn’t just for CI or producing optimized distributables) as there was no path to directly load these components in a browser, and I’d argue also led to a state where components were being composed with an increasing and ridiculous number of dependencies, as the burden of resolving and flattening dependencies fell to to the toolchain.

ES modules aren’t any sort of silver bullet here, but it does show a future where some of this complexity can be rolled back, the toolchain only needs to handle and be invoked for specific cases, and there is less impedance in working with frontend code. We’re not there yet, but I’m hopeful and as such I’ve adopted ES modules in projects where I’ve been able to.

A look at GraphPaper

GraphPaper was the first project where I committed to ES modules for the codebase. At the time, support for ES modules was limited in both Node and browsers (typically incomplete support and put behind a feature flag), so for both development work and producing distributables, I used rollup.js + Babel to produce an IIFE module. This worked well, though it’s a pain to do a build every for every code change I want to see in the browser. I also remember this being pretty easy to setup initially, but the package.json became more convoluted with Babel 6, when everything was split into smaller packages (I understand the rationale, but the developer experience is horrible and pushes the burden of understanding various babel components to consumers).

Structuring the module

Structuring was fairly simple. Everything that was meant to be accessible by consumers was declared in a single file (GraphPaper.js), declared via export statements (i.e. it was just a file with a bunch of export * from … statements). This file also served as the input for rollup.js:

{ input: 'src/GraphPaper.js', output: { format: 'iife', file: 'dist/graphpaper.min.js', name: 'GraphPaper', sourcemap: true }, plugins: [ babel(babelConfig), ], }

In a modern browser, it would be possible to import the GraphPaper ES module directly, like so:

<script type="module"> import * as GraphPaper from '../src/GraphPaper.js'; ... </script>

However, the way dependent web workers are built and encapsulated makes this impossible (explained here).

Another problem, but one that’s fixable, is that I wrote import and export statements without file extensions. For browsers, this leads to an issue as the browser will just request what’s declared in the statement and not append any file extension (so a statement like import { LineSet } from './LineSet'; results in a request to the server for ./LineSet, not ./LineSet.js, as the file is named. Moving forward, the recommendation to use the .mjs extension and explicitly specifying the extension in import statements seems to be a good idea; in addition to addressing the issue with browser requests, when working in Node .mjs files will automatically be treated as ES modules.

Pushing aside the web worker issue, we can see a future where rollup.js isn’t necessary, or at least not necessary for producing browser-compatible (e.g. IIFE) modules. It’s role can be limited to concatenation and orchestrating optimizations (e.g. minification) for distributables. Similarly, for Babel, it’s role can be reduced or eliminated. As support for newer ES features (particularly those in ES6 and ES7) continues to improve across systems (browsers, Node, etc.), and users adopt these systems, transpilation won’t be as necessary. The exception is the case where developers want to use the very latest ES features, but I think we’re quickly approaching a point of diminishing returns here, especially relative to the cost of toolchain complexity.

Testing with jasmine-es6, moving to Ava

For testing, I found jasmine-es6 to be one of the simpler ways to test ES modules at the time. Ava existed, but I remember running into issues getting it working. I remember also toying with Jest at some point and also running into issues. In the end, jasmine-es6 worked well, I never had issues importing and writing tests for a module. Here’s a sample test from the codebase:

import { Point } from '../src/Point' import { Line } from '../src/Line' import { LineSet } from '../src/LineSet' describe("LineSet constructor", function() { it("creates LineSet from Float64Array coordinates", function() { const typedArray = Float64Array.from([1,2,3,4,5,6,7,8]); const ls = new LineSet(typedArray); const lineSetArray = ls.toArray(); expect(ls.count()).toBe(2); expect(lineSetArray[0].isEqual(new Line(new Point(1, 2), new Point(3, 4)))).toBe(true); expect(lineSetArray[1].isEqual(new Line(new Point(5, 6), new Point(7, 8)))).toBe(true); }); });

jasmine-es6 has and continues to work really well despite being deprecated. I’ll likely adopt and reformat the tests to Ava at some point the future. I’ve played around with it again recently and it was a much smoother experience, it’s also better supported and I like the simpler syntax around tests more-so than the Jasmine syntax. I’m looking to do this when Node has stable support for ES modules, as this would mean not worrying about pulling in and configuring Babel for running tests (though it’ll likely still be around for rollup.js).


Overall, it’s been fairly smooth working with ES modules and it looks like things will only improve in the future. Equally exciting is the potential reduction in toolchain complexity that comes with better support for ES modules.

  • Support for ES modules continues to improve across libraries, browsers, and Node
  • It’s probably a good idea to use the .mjs file extension
  • Rollup.js is still needed for now to make browser-compatible (e.g. IIFE) modules, but will likely take on a more limited role in the future (concatenation & minification)
  • Better support for ES6 and ES7 features across the board will mean that Babel, and transpilation in general, won’t be as necessary