semi/signal

How to local dev

Local developer environments can take many forms but for non-trivial web applications I’m still fond of Vagrant to spin up a VM that is “close enough” to production environments. Typically, I will create a Vagrant box and map a hostname to the IP of the box via an entry in the host system’s hosts file (I use vagrant-hostmap for this, as DHCP is used to avoid conflicts and that means the IP address change often).

Does HTTPS matter for local dev?

Probably not. I went down this particular rabbit hole as I was doing some experiments with server-side events and noticed there’s a limitation on open connection when not using HTTP/2.

So, I thought it would be a good idea to get HTTP/2 working in my local dev environment, which was using Apache and PHP. This is fairly straightforward to setup on Apache. However, the browser (Chrome) was not happy with. Currently, browsers only support HTTP/2 over TLS (h2) and there’s is no support for HTTP/2 Cleartext (h2c). There some good reasons for this.. but many of these reasons are addressing concerns on the public web, for other use-cases (e.g. local dev) there’s additional complexity to support h2 with minimal benefits.

Scripting

The code in the sections below is Bash for a provisioning script (bootstrap.sh), intended to run when the Vagrant is provisioned. The code is such that the environment of the box is reproducible (after destroying and re-creating) and there is no need to re-adjust configuration on the host machine (comes into play when dealing with certificates).

The Vagrantfile looks something like this:

… with the bootstrap script and related assets (e.g. Apache conf files) within the localdev folder.

Supporting h2: disable prefork

First, disable the prefork module, it doesn’t play well with HTTP/2:

In prefork, mod_http2 will only process one request at at time per connection. But clients, such as browsers, will send many requests at the same time. If one of these takes long to process (or is a long polling one), the other requests will stall.

Supporting h2: enable HTTP/2 modules and configuration

Enable modules and configuration for HTTP/2:

The http2.conf file contains the following:

Required modules and configuration are now enabled. For a given VirtualHost definition, you can add Protocols h2 h2c http/1.1.

e.g.:

Supporting h2: minica

For HTTPS TLS certificate generation, minica is awesome and simple to use. However, it is only distributed as source, so you’ll need to install git to checkout the repo and golang to compile the source.

Here a /tools folder is created, the minica repo is cloned within, and the source is compiled.

A /certs folder is also created as a location for generated certificates.

Supporting h2: generate root CA certificate

Once compiled, run minica to generate a root CA certificate (minica.pem) and key (minica-key.pem), which can be installed on the host to avoid browsers showing that the certificate is invalid or untrusted. This shouldn’t be done in the Vagrant box provisioning script, as you’ll get new root certificates every time the box is provisioned. Instead, SSH into the box, run, and copy the files to a secure location.. or run minica somewhere else (e.g. host machine or another machine). ./minica --domains "test.localdev.com"

minica requires a domain argument but the need is really for the minica.pem and minica-key.pem files at this point, not the site certificate. Generation of the site certificate is something that should be done within the provisioning script.

Supporting h2: generate site certificate

Within the provisioning script:

There’s a 2 things to note here:

• To avoid having to generate new certificates for different subdomains, a wildcard cert is generated (*.localdev.com)
• An existing root CA certificate, and key, is expected

Supporting h2: use site certificate

Update the VirtualHost definition to reference the site certificate and key:

Trusting minica certs on the host

Finally, to avoid the browser showing a warning that the site’s certificate is invalid or insecure, add the root certificate as a trusted root CA authority on the host machine. Martin Widmann has some nice documentation on how to do this for different operating systems.

Verify h2 protocol in the browser

With the site up, Chrome and Firefox dev tools should now show that traffic is being served via the h2 protocol.

The Protocol column isn’t visible by default, so you may have to enable it (right click on visible header → select Protocol).

Visibility into the performance of backend components can be invaluable when it comes to spotting and understanding service degradation, debugging failures, and knowing if and where optimization is needed. There’s a host of collection agents, aggregators, and visualization tools to handle metrics, but just breaking down and looking at what happens during an HTTP request can offer a lot of insight into how components are performing. This is why I’m pretty excited about the the HTTP Server-Timing header, it works well as a lightweight mechanism to surface performance metrics, especially now that it’s read and graphed by Chrome Devtools (and, perhaps sometime soon, by Firefox Devtools as well).

An HTTP response with the Server-Timing header

The following code snippet shows an Illuminate/Http/Response from a controller that PUTs an image into an Amazon S3 bucket.

return response()
    ->json(
        [],
        StatusCode::STATUS_OK,
        [
            'Server-Timing' => 's3-io;desc="Image upload to S3";dur=' . calculateTimeToPut(),
        ]
    );

Let’s assume the calculateTimeToPut() function returns 5500 (i.e. 5500 milliseconds to PUT the image onto S3), and the response header looks something like this:

Each metric is a group composed of 3 pieces, with each piece delimited by a semicolon:

• Metric Name (required)
• Metric Description
• Metric Value

Multiple metrics can be surfaced by separating each group with a comma.

return response()
    ->json(
        [],
        StatusCode::STATUS_OK,
        [
            'Server-Timing' => 
                's3-io;desc="Image upload to S3";dur=' . calculateTimeToPut() . 
                ',' . 
                'db-io;desc="DB update of entity";dur=' . calculateTimeToUpdate()
        ]
    );

(The above code is a bit simplistic, you’d likely want to better way to store and group metrics, then do a final transformation to construct the Server-Timing string when it’s time to send the HTTP response)

Surfacing in DevTools

Surfacing metrics in an HTTP response is not something terribly complex and I’m sure most could devise other ways to do it, but one reason Server-Timing is a bit more attractive vs a custom solution is the out-of-the-box support within Chrome DevTools.

Firefox Devtools will likely follow suit (hopefully?) in the near future.

The PerformanceServerTiming interface

Server-Timing metrics can also be surfaced via the PerformanceServerTiming interface, from MDN:

In addition to having Server-Timing header metrics appear in the developer tools of the browser, the PerformanceServerTiming interface enables tools to automatically collect and process metrics from JavaScript.

This opens up some interesting possibilities as it enables collecting metrics via a frontend script (as is already done for a lot of product metrics via services like Google Analytics), rather than a backend collector mechanism. While not ground-breaking, the standardization around PerformanceServerTiming may allow for greater adoption and acceptance of this collection pattern.