Posts Tagged ‘computer graphics’

Using feColorMatrix to dynamically recolor icons (part 1, single-color icons)

I’ve been experimenting with using feColorMatrix as an elegant way to dynamically color/re-color SVG icons. Here I’ll look at working with single-color icons.

Working directly with the SVG markup

Changing the stroke and/or fill colors of the SVG elements directly can be a good solution in many cases, but it requires:

  • Placing the SVG markup into the document to query and modify the appropriate elements when a color update is needed (note that this option isn’t viable if you need to place the icon in an <img> tag or it needs to be placed as a background-image on an element, as you can’t reference the SVG element in such cases)
  • Treating the SVG markup as a templated, Javascript, string to make a data-URI, and re-making it when a color update is needed

By using the color transformation matrix provided by feColorMatrix these restrictions go away and we also get back the flexibility of using external files.

Icon color

Keep in mind, we’re only dealing with single-color icons. What color is used doesn’t technically matter, but black is a nice basis and in an actual project, black is beneficial, as you’re able to open your the icon files in an editor or browser and actually see it.

black colored icon

A black pixel within the icon can then be represented by the following vector:

black pixel as column vector

Note that the alpha component may vary due to antialiasing (to smooth out edges) or some translucency within the icon.

The color transformation matrix

feColorMatrix allows you to define a 5×4 transformation matrix. There’s a lot you can do with that, but note that the last column of the matrix is essentially an additive component for each channel (see matrix-vector multiplication), so in that column we enter the desired R, G, B values from top to bottom, which will be added to the zeros in the input vector. Next, we want to preserve the alpha component from the input vector, so the fourth column of the matrix becomes [0, 0, 0, 1]T and the fourth row of the last column is zero, as we don’t want to add anything to the alpha component.

color transformation matrix, [[0, 0, 0, 0, R], [0, 0, 0, 0, G], [0, 0, 0, 0, B], [0, 0, 0, 1, 0]][(0), (0), (0), (A_(src))] = [(R), (G), (B), (A_(src))]

The matrix-vector multiplication gives a new vector (that defines the output pixel) with the entered R, G, B values and the alpha value from the source pixel.

Representing the matrix within an feColorMatrix element is straightforward…

<feColorMatrix in="SourceGraphic" type="matrix"
values="0 0 0 0 R
0 0 0 0 G
0 0 0 0 B
0 0 0 1 0"

… just plug in values for R, G, B.

Applying the color transformation

The color transformation matrix can be applied to an element by wrapping it in an SVG filter element and referencing the filter via the CSS filter property.

With Javascript, the values attribute of the feColorMatrix element can be updated dynamically. The color change will, in turn, be reflected in any elements referencing the SVG filter.

<!DOCTYPE html>
The values of the color matrix defines the color transformation what will be applied.
Here we just setup the elements and define an identity matrix. We'll modify the matrix via Javascript code
to define an actual color transformation.
<svg style="width:0; height:0; margin:0; padding:0; border:none;">
color-interpolation-filters="sRGB" id="colorTransformFilter">
in="SourceGraphic" type="matrix"
values="1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0"


Element with an SVG icon that we want to colorize
Note: that the color transformation is applied to everything not only to the background, but everything
within the element as well.

Typical solution to to isolate background stuff to it's own div and use another div for contents

<div id="logo-colored"
background-color: transparent;
background-image: url(logo.svg);
background-position: center center;
background-repeat: no-repeat;
background-size: cover;

style="color:#fff">Testing 1 2 3...</p>

* A little helper function to update the color transformation matrix
* @param {Number} _r
* @param {Number} _g
* @param {Number} _b
const setPrimaryColor = function(_r, _g, _b) {

const rScaled = _r / 255.0;
const gScaled = _g / 255.0;
const bScaled = _b / 255.0;

const feColorMatrixElem = document.getElementById('colorTransformFilter').getElementsByTagName('feColorMatrix')[0];
`0 0 0 0 ${rScaled}
0 0 0 0 ${gScaled}
0 0 0 0 ${bScaled}
0 0 0 1 0`

// Set/update color transformation matrix
setPrimaryColor(129, 0, 0);


The code will take a black-colored icon and re-color it to [129, 0, 0], as seen below:


This technique provides a lot of flexibility, but it’s not without it’s limits.

  • For icons applied as background-images, the CSS filter property isn’t ideal. CSS filter will effect not only the element it’s applied to, but all child elements as well. Note the “Testing 1 2 3…” paragraph is re-colored in the demo above.
  • As is the case with mixing the CSS filter property and the SVG filter element, effects governed by the CSS transition property won’t work.

Similar techniques

  • For shifting between colors, the hue-rotate() CSS filter function can be a solution. However, in practice, I don’t find this intuitive and color changes are rarely just hue rotations.
  • A more limited case, transitioning a colored icon to white, can be achieved with 2 CSS filter functions, brightness(0) and invert(100%).
  • You can do crazier things by trying to compute and fit a solution to the hue-rotation, saturation, sepia, and invert filter functions; however this is both complex to grasp and produces inexact/approximate color matches.
  • An SVG filter using feComponentTransfer should work, but I don’t find it as intuitive to work with.


If you want to interactively play around with feColorMatrix, check out SVG Color Filter Playground.

Encoding MP4s in the browser

Is this possible?

Given that it’s relatively easy to access a camera and capture frames within a browser, I began wondering it there was a way to encode frames and create a video within the browser as well. I can see a few benefits to doing this, perhaps the biggest being that you can move some very computationally expensive work to front-end, avoiding the need to setup and scale a process to do this server-side.

I searched a bit and first came across Whammy as a potential solution, which take a number of WebP images and creates a WebM video. However, only Chrome will let you easily get data from a canvas element as image/webp (see HTMLCanvasElement.toDataURL docs). The non-easy way is to read the pixel values from the canvas element and encode them as WebP. However, I also couldn’t find any existing JS modules that did this (only a few NodeJS wrappers for the server-side cwebp application) and writing an encoder was a much bigger project that I didn’t want to undertake.

The other option I came across, and used, was ffmpeg.js. This is a really interesting project, it’s a port of ffmpeg via Emscripten to JS code which can be run in browsers that support WebAssembly.

Grabbing frames

My previous post on real-time image processing covers how to setup the video stream, take a snapshot, and render it to a canvas element. To work with ffmpeg.js, you’ll additionally need the frame’s pixels from the canvas element as a JPEG image, represented as bytes in a Uint8Array. This can be done as follows:

var dataUri = canvas.toDataURL("image/jpeg", 1);
var jpegBytes = convertDataURIToBinary(dataUri);

convertDataURIToBinary() is the following method, which will take the data-uri representation of the JPEG data and transform it into a Uint8Array:

function convertDataURIToBinary(dataURI) {
var base64 = dataURI.substring(23);
var raw = window.atob(base64);
var rawLength = raw.length;

var array = new Uint8Array(new ArrayBuffer(rawLength));
for (i = 0; i < rawLength; i++) {
array[i] = raw.charCodeAt(i);
return array;

FYI, this is just a slight modification of a method I found in this gist.

Note that I did not use PNG images due to an issue in the current version of ffmpeg.js (v3.1.9001).

Working with ffmpeg.js

ffmpeg.js comes with a Web Worker wrapper (ffmpeg-worker-mp4.js), which is really nice as you can run “ffmpeg –whatever” by just posting a message to the worker, and get the status/result via messages posted backed to the caller via Worker.onmessage.

var worker = new Worker("node_modules/ffmpeg.js/ffmpeg-worker-mp4.js");
worker.onmessage =
function (e) {
var msg =;

switch (msg.type) {
case "ready":
'mp4 worker ready');
case "stdout":
case "stderr":

case "done":
var blob = new Blob([[0].data], {

// ...

case "exit":
"Process exited with code " +;

Input and output of files is handled by MEMFS (one of the virtual file systems supported by Emscripten). On the “done” message from ffmpeg.js, you can access the output files via the array (shown above). Input files are specified via an array in the call to worker.postMessage (shown below).

TOTAL_MEMORY: 268435456,
data: jpegBytes
arguments: [
"-r", "60", "-i", "input.jpeg", "-aspect", "16/9", "-c:v", "libx264", "-crf", "1", "-vf", "scale=1280:720", "-pix_fmt", "yuv420p", "-vb", "20M", "out.mp4"]


With a bunch of frames captured from the video stream, I began pushing them through ffmpeg.js to encode a H.264 MP4 at 720p, and things started to blow up. There were 2 big issues:

  • Video encoding is no doubt a memory intensive operation, but even for a few dozen frames I could never give ffmpeg.js enough. I tried playing around with the TOTAL_MEMORY prop in the worker.postMessage call, but if it’s too low ffmpeg.js runs out of memory and if it’s too high ffmpeg.js fails to allocate memory.
  • Browser support issues. Support issues aren’t surprising here given that WebAssembly is still experimental. The short of it is: things work well in Chrome and Firefox on desktop. For Edge or Chrome on a mobile device, things work for a while before the browser crashes. For iOS there is no support.

Hacking something together

The browser issues were intractable, but support on Chrome and Firefox was good enough more me, and I felt I could work around the memory limitations. Lowering the memory footprint was a matter of either:

  • Reducing the resolution of each frame
  • Reducing the number of frames

I opted for the latter. My plan was to make a small web application to allow someone to easily capture and create time-lapse videos, so I had ffmpeg.js encode just 1 frame to a H.264 MP4, send that MP4 to the server, and then use ffmpeg’s concat demuxer on the server-side to progressively concatenate each individual MP4 file into a single MP4 video. What this enables is for the more costly encoding work to the done client-side and the cheaper concatenation work to be done server-side.

Time Stream was the end result.

Here’s a time-lapse video created using an old laptop and a webcam taped onto my balcony:

This sort of hybrid solution works well. Overall, I’m happy with the results, but would love the eliminate the server-side ffmpeg dependency outright, so I’m looking forward to seeing Web Assembly support expand and improve across browsers.

More generally, it’s interesting to push these types of computationally intensive tasks to the front-end, and I think it presents some interesting possibilities for architecting and scaling web applications.

Brute-force convex hull construction

I’ve been experimenting a bit with convex hull constructions and below I’ll explain how to do a brute-force construction of a hull.

It’s worth noting up-front that the brute-force method is slow, O(n3) worst case complexity. So why bother? I think there are a few compelling reasons:

  • The brute-force method expresses the fundamental solution, which gives you the basic building blocks and understanding to approach more complex solutions
  • It’s faster to implement
  • It’s still a viable solution when n is small, and n is usually small.

What is a convex hull?

You can find a formal definition on Wikipedia. Informally, and specific to computational geometry, the convex hull is a convex polygon in which all points are either vertices of said polygon or enclosed within the polygon.

Brute-force construction

  • Iterate over every pair of points (p,q)
  • If all the other points are to the right (or left, depending on implementation) of the line formed by (p,q), the segment (p,q) is part of our result set (i.e. it’s part of the convex hull)

Here’s the top-level code that handles the iteration and construction of resulting line segments:

* Compute convex hull
var computeConvexHull = function() {
"--- ");

for(var i=0; i<points.length; i++) {
for(var j=0; j<points.length; j++) {
if(i === j) {

var ptI = points[i];
var ptJ = points[j];

// Do all other points lie within the half-plane to the right
var allPointsOnTheRight = true;
for(var k=0; k<points.length; k++) {
if(k === i || k === j) {

var d = whichSideOfLine(ptI, ptJ, points[k]);
if(d < 0) {
allPointsOnTheRight =

if(allPointsOnTheRight) {
"segment " + i + " to " + j);
var pointAScreen = cartToScreen(ptI, getDocumentWidth(), getDocumentHeight());
var pointBScreen = cartToScreen(ptJ, getDocumentWidth(), getDocumentHeight());
drawLineSegment(pointAScreen, pointBScreen);


The “secret sauce” is the whichSideOfLine() method:

* Determine which side of a line a given point is on
var whichSideOfLine = function(lineEndptA, lineEndptB, ptSubject) {
return (ptSubject.x - lineEndptA.x) * (lineEndptB.y - lineEndptA.y) - (ptSubject.y - lineEndptA.y) * (lineEndptB.x - lineEndptA.x);

This is a bit of linear algebra derived from the general equation for a line.

The result represents the side of a line a point is one, based on the sign of the result. We can check if the point is on the left or on the right, it doesn’t matter as long as there is consistency and the same check is done for all points.

How it looks

I made a few diagrams to show the first few steps in the algorithm, as segments constituting the convex hull are found. The shaded area represents our success case, where all other points are to the right of the line formed by the points under consideration. Not shown are the failure cases (i.e. one or more points are on the left of the line formed by the points under consideration).

convex hull construction, brute force, step 1

convex hull construction, brute force, step 1

convex hull construction, brute force, step 1

Code and Demo

You can play around with constructing a hull below by double-clicking to add vertices.

You can find the code on GitHub.

Post-process shaders in glfx

Pushed an update to glfx to allow for post-process shading. When a post-process shader is defined, the scene is rendered to a screen-space quad (the size of the viewport), and that quad is then rendered to the viewport with the post-process shader applied.

The shader is loaded (asynchronously) like any other:

glfx.shaders.load('screenspace.fs', "frag-shader-screenspace",;

Once loaded, we create the shader program, and get locations for whatever variables are used. The vertex shader isn’t anything special, it just transforms a vertex by the model-view and projection matrices, and passes along the texture coordinates.

glfx.whenAssetsLoaded(function() {

var postProcessShaderProgram = glfx.shaders.createProgram([glfx.shaders.buffer['vert-shader-basic'], glfx.shaders.buffer['frag-shader-screenspace']],
function(_shprog) {

// Setup variables for shader program
_shprog.vertexPositionAttribute =, "aVertexPosition");
_shprog.pMatrixUniform =,
_shprog.mvMatrixUniform =,
_shprog.textureCoordAttribute =,

_shprog.uPeriod =,
_shprog.uSceneWidth =,
_shprog.uSceneHeight =,



We then tell glfx to apply our post-process shader program:


This call will result in different rendering path, which renders the scene to a texture, applies that texture to a screen-space quad, and renders the quad with the post-process shader.

Here is the shader for screenspace.fs, used in the demo shown above:

precision mediump float;

uniform float uPeriod;
uniform float uSceneWidth;
uniform float uSceneHeight;
uniform sampler2D uSampler;        
varying vec2 vTextureCoord;

void main(void) {

vec4 sum = vec4( 0. );
float blurSampleOffsetScale = 2.8;
float px = (1.0 / uSceneWidth) * blurSampleOffsetScale;
float py = (1.0 / uSceneHeight) * blurSampleOffsetScale;

vec4 src = texture2D( uSampler, ( vTextureCoord ) );

sum += texture2D( uSampler, ( vTextureCoord + vec2(-px, 0) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(-px, -py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(0, -py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(px, -py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(px, 0) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(px, py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(0, py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(-px, py) ) );
sum += src;

sum = sum / 9.0;

gl_FragColor = src + (sum * 2.5 * uPeriod);


Note that it requires a few uniforms to be supplied to it, we use the glfx.scene.onPostProcessPreDraw() callback to setup the variables (before the post-processed scene is drawn):

var timeAcc = 0;
glfx.scene.onPostProcessPreDraw =
function(tdelta) {

timeAcc += tdelta;
var timeScaled = timeAcc * 0.00107;

if(timeScaled > 2.0*Math.PI) {
timeScaled = 0;
timeAcc = 0;

var period = Math.cos(timeScaled);, period + 1.0);,;,;

What we’re doing is using the scene rendering time deltas to generate a periodic/sinusoidal wave. This results in the pulsing brightness/fading effect of the scene. The brightness effect itself is done by adding the source pixel to a blurred + brightened version of itself. The blurring allows for the soft fade in and fade out.

GLSL variable qualifiers

I’ve been playing around with WebGL shader code recently and found this bit on variable prefixes helpful, particularly in the explanation of the variable qualifiers:

  • Attribute: data provided by buffers
  • Uniform: inputs to the shaders
  • Varying: values passed from a vertex shader to a fragment shader and interpolated (or varied) between the vertices for each pixel drawn

Something important to keep in mind is that this relates to the OpenGL ES Shading Language, Version 1.00, which is (unfortunately) what’s currently supported by WebGL.

A WebGL implementation must only accept shaders which conform to The OpenGL ES Shading Language, Version 1.00 [GLES20GLSL], and which do not exceed the minimum functionality mandated in Sections 4 and 5 of Appendix A.

Attribute and Varying were part of early versions of, OpenGL-supported, GLSL, but are deprecated as of OpenGL 3.0 / GLSL 1.30.10, and replaced with more generic constructs:

  • in is for input from the previous pipeline stage, i.e. per vertex (or per fragment) values at most, per primitive if using glAttribDivisor and hardware instanciation
  • out is for output to the next stage

Real-time image processing on the web

A while ago I began playing around with grabbing a video stream from a webcam and seeing what I could do with the captured data. Capturing the video stream using the navigator.getUserMedia() method was straightforward, but directly reading and writing the image data of the video stream isn’t possible. That said, the stream data can be put onto a canvas using CanvasRenderingContext2D.drawImage(), giving you to ability to read the pixel data. When it comes to writing visual data, a few options are available.

var videoElem = document.querySelector('video');

// Request video stream
navigator.getUserMedia({video: true, audio: false},

function(_localMediaStream) {
videoStream = _localMediaStream;
videoElem.src = window.URL.createObjectURL(_localMediaStream);

function(err) {
'navigator.getUserMedia error' + err);

var videoElem = document.querySelector('video');
var canvas = document.querySelector('canvas');
var ctx = canvas.getContext('2d');


// put snapshot from video stream into canvas
ctx.drawImage(videoElem, 0, 0);

You can read and write to the <canvas> element, so hiding the <video> element with the source data and just showing the <canvas> element is an option, but the CanvasRenderingContext2D.drawImage() call is expensive; looking at the copied stream on the <canvas> element there is, very noticeable, visual lag. Another reason to avoid this option is that the frequency at which you render (e.g. 30 FPS), isn’t necessarily the frequency at which you’d want to grab and process image data (e.g. 10 FPS). The disassociation allow you to keep the video playback smooth, for a better user experience, but more effectively utilize CPU cycles for the image processing. At least in my experiences so far, a small delay in the visual feedback from the image processing is acceptable and looks perfectly fine intermixed with the higher-frequency video stream.

Throwing aside reading and writing to just the <canvas> element, alternative options all involve showing the <video> element with the webcam stream and placing visual feedback on top of the video pixels. A few ideas:

  • Write pixel data to another canvas and render it on top of the <video> element
  • Render SVG elements on top of the <video> element
  • Render DOM elements (absolutely positioned) on top of the <video> element

The third option is an ugly solution, but it’s fast to code and thus allows for quick prototyping. The demo and code below shows a quick demo I slapped together using <div> elements as markers for hot spots, in this case bright spots, within the video.

<!DOCTYPE html>
title>Webcam Cap</title>
meta charset="UTF-8">
meta name="viewport" content="width=device-width, initial-scale=1.0">

style type="text/css">
* { margin:0; padding:0; border:none; }


video style="width:640px; height:480px;" width="640" height="480" autoplay></video>
canvas style="display:none; width:640px; height:480px;" width="640" height="480"></canvas>

div class="ia-markers"></div>

script type="text/javascript">

navigator.getUserMedia = (navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia);

if ( typeof navigator.getUserMedia !== 'undefined' ) {

var videoElem = document.querySelector('video');
var canvas = document.querySelector('canvas');
var ctx = canvas.getContext('2d');
var videoStream = null;
var snapshotIntv = null;

var width = 640;
var height = 480;

// Request video stream
navigator.getUserMedia({video: true, audio: false},

function(_localMediaStream) {
videoStream = _localMediaStream;
videoElem.src = window.URL.createObjectURL(_localMediaStream);

// Take a snapshot of the video stream 10ms
snapshotIntv = setInterval(function() {
}, 100);


function(err) {
'navigator.getUserMedia error' + err);


// Take a snapshot from the video stream
function processSnapshot() {

// put snapshot from video stream into canvas
ctx.drawImage(videoElem, 0, 0);

// Clear old snapshot markers
var markerSetParent = (document.getElementsByClassName('ia-markers'))[0];
markerSetParent.innerHTML =

// Array to store hotzone points
var hotzones = [];

// Process pixels
var imageData = ctx.getImageData(0, 0, width, height);
for (var y = 0; y < height; y+=16) {
for (var x = 0; x < width; x+=16) {
var index = (x + y * imageData.width) << 2;

var r =[index + 0];
var g =[index + 1];
var b =[index + 2];

if(r > 200 && g > 200 && b > 200) {

// Add new hotzone elements to DOM
for(var i=0; i<hotzones.length; i++) {
var x = hotzones[i][0];
var y = hotzones[i][1];

var markerDivElem = document.createElement("div");
'style', 'position:absolute; width:16px; height:16px; border-radius:8px; background:#0f0; opacity:0.25; left:' + x + 'px; top:' + y + 'px');
markerDivElem.className =


else {
'getUserMedia() is not supported in your browser');



WebGL on a high-DPI display

Dealing with WebGL on a high-DPI display isn’t too difficult, but it does require an understanding of device pixels vs CSS pixels. Elements on a page automatically upscale on a high-DPI display, as dimensions are typically defined with CSS, and therefore defined in units of CSS pixels. The <canvas> element is no exception. However, upscaling a DOM element doesn’t mean that the content within the element will be upscaled or rendered nicely – this is why non-vector content can appear blurry on higher resolutions. With WebGL content, not only will it appear blurry, but the viewport will likely be clipped as well, due to the viewport being incorrectly calculated using CSS pixel dimensions.

With WebGL everything is assumed to be in units of device pixels and there is no automatic conversion from CSS pixels to device pixels. To specify the device pixel dimensions of the <canvas>, we need to set the width and height attributes of the element:

CSS Pixels vs Device Pixels on canvas

I like to compute and set the attributes automatically using window.devicePixelRatio.
In glfx I do the following (passing in _canvasWidthCSSPx, _canvasHeightCSSPx):

// Get devicePixelRatio
glfx.devicePixelRatio = window.devicePixelRatio || 1;

// Set the width,height attributes of the canvas element (in device pixels)
var _canvasWidthDevicePx = _canvasWidthCSSPx * glfx.devicePixelRatio;
var _canvasHeightDevicePx = _canvasHeightCSSPx * glfx.devicePixelRatio;    
"width", _canvasWidthDevicePx);
"height", _canvasHeightDevicePx);

// Set viewport width,height based on dimensions of canvas element = _canvasWidthDevicePx; = _canvasHeightDevicePx;        

Reference: HandlingHighDPI

Where we live

Using a number of technologies I’ve been playing around with recently, I began working on a 3D visualization of the Earth, plotting every city, creating a pointillism-styled representation of the planet. Below is the result along with an overview of how I produced the rendering.

Getting the data

I extracted all cities with a population of at least 100,000 people from the MySQL GeoNames database using the following query:

SELECT `id`,`name`,`latitude`,`longitude`,`population`,`timezone`
FROM geonames.cities
WHERE population >= 100000 AND feature_class = 'P';

… and put the results into a JS array.

Creating a 3D model to represent each city

I created this hexagonal model in Blender, exported it to a Wavefront OBJ file, and ran the OBJ file through the Wavefront OBJ to JSON converter I wrote. Note that the model is facing the z-axis to match WebGL’s (and OpenGL’s) default camera orientation: facing down the negative z-axis.

Convert longitude and latitude to a 3D position

Converting a geodetic longitude, latitude pair to a 3D position involves doing a LLA (Longitude Latitude Altitude) to ECEF (Earth-Centered, Earth-Fixed) transformation. The code below implements this transform, converting the longitude and latitude of every city pulled from the GeoNames database into a 3D coordinate where we can render the hexagonal representation of the city.

function llarToWorld(lat, lon, alt, rad)
    lat = lat * (Math.PI/180.0);
    lon = lon * (Math.PI/180.0);

var f = 0; //flattening
var ls = Math.atan( Math.pow((1.0 - f),2) * Math.tan(lat) ); // lambda

var x = rad * Math.cos(ls) * Math.cos(lon) + alt * Math.cos(lat) * Math.cos(lon)
var y = rad * Math.cos(ls) * Math.sin(lon) + alt * Math.cos(lat) * Math.sin(lon)
var z = rad * Math.sin(ls) + alt * Math.sin(lat)
return [x,z,-y];            

There are 2 items worth noting:

  • The transformation (and function above) involve a 4th parameter, radius which is the radius of the ellipsoid (or sphere, in this case, as flattening=0) into which the transformation is done. I have it set as a fixed constant, as I’m primary concerned with an approximate visual representation, but the MathWorks page describes the actual computation.
  • The ECEF (Earth-Centered, Earth-Fixed) coordinate system has the z-axis pointing north, not the y-axis, so the z and y values need to be swapped to produce a coordinate corresponding to WebGL’s default camera orientation. In addition, as WebGL has a right-handed coordinate system (so the default camera orientation is one where it’s pointing down the negative z-axis), the z coordinate is negated so the point doesn’t wind up behind the camera.

Orient all cities to face the origin

Getting each of the hexagonal models to face the origin involved a bit of math:

  • Calculating the axis about which the rotation should occur by, first, computing a vector from the origin to the 3D position of the model (lookAt), and taking the cross product between lookAt and the z-axis (as we’re rotating toward the z-axis).
  • Calculating the angle of rotation (the angle between the z-axis and lookAt) by computing the dot product between lookAt and the z-axis, then taking the acos of the dot product.

There’s some additional code to handle cases where points lie on the on the z-axis (where the cross product gives the zero vector) and also to return a matrix representation of the rotation.

function lookAtOrigin(v)
// compute vector from origin
var lookAt = vec3.create([v[0], v[1], -v[2]]);

// reference axis
var refAxis = vec3.create([0,0,-1]);

// computate axis of rotation
var rotAxis = vec3.create(lookAt);
vec3.cross(rotAxis, refAxis);

// compute angle of rotation
var rotAngRad = Math.acos(, refAxis));

// special cases...
if(rotAxis[0] == 0 && rotAxis[1] == 0 && rotAxis[2] == 0) {
if(lookAt[2] > 0) {
rotAxis = vec3.create([1,0,0]);
rotAngRad = Math.PI;
else {
rotAxis = vec3.create([1,0,0]);
rotAngRad = 0;

// compute and return a matrix with the rotation
var ret = mat4.identity();
mat4.rotate(ret, rotAngRad, rotAxis);

return ret;

Render the scene

Using glfx, I pulled everything together, also adding a bit of code to rotate the camera and do some pseudo-lighting in the pixel shader by alpha blending colors based on depth. All the code can be found in the webgl-globe repository on bitbucket.

Entering the world of high-DPI displays

With a Retina iPad and my recent purchase of a Yoga 2 laptop with a “Quad HD” display (3200×1800) I’ve been dragged into the world of high-DPI (more precisely PPI) displays. For years, DPI was “standard” at either 96dpi (Windows) or 72dpi (Mac OS), with a logical/software pixel being equivalent to a hardware pixel on the display device. A higher resolution monitor meant the content on your display got a bit smaller but you gained a couple more thousand pixels to work with, but the recent and massive increases in pixel densities seems to be the end of the 1:1 mapping between software and hardware pixel references. Below are a few notes on my experiences dealing with high-DPI displays and content so far.

  • Windows 8.1 support is terrible. Both application support and operating system support for high-DPI displays is abysmal. See the post Living a High-DPI desktop lifestyle can be painful by Scott Hanselman which reflects many of the issues I’ve encountered with my Yoga 2 as well. There’s a large list of issues: for non-DPI aware applications Windows scale text but not icons and layout, applications lie about being DPI-aware and are rendered too small, and some applications simply crash (TourtiseSVN’s diff… no clue why, too many pixels?!). It’s easy to wag a finger at application developers, but legacy support is clearly something the operating system needs to handle. In addition, despite Windows 8.1 touting automatic, per-monitor, DPI detection, it’s all based off of the DPI of a “primary monitor” and content on the other monitors is scaled to match. So dragging a window for an application from the Yoga 2 display across to a HD/96dpi monitor results in the window being scaled down (and visibly blurry). Worse, all applications undergo the same treatment – so if you’re thinking you can just use non-DPI aware applications on an external display until support comes around, guess again.
    ArsTechnica did a piece mentioning this issue in particular, and Windows 8.1’s high-DPI support is general.
  • Web support is only half-way there. The best thing done to support high-DPI displays was defining the CSS2 reference pixel to be independent of hardware pixels. Beyond that you have media queries and higher resolution background images, but there’s still no good way to specify alternate foreground images, though the <picture> element may gain support soon. In general, outside of CSS things gets messy, as is the case with a high-DPI <canvas>.
    One problem with the web that I don’t see a proposed solution for is handling low-resolution image assets for which you can’t get a higher resolution version. This is a problem I face with this blog. There’s a lot of images (old screenshots, low-resolution photos, etc.) for which I can’t get a 2x, 4x, etc., higher resolution version and there’s no way to prevent upscaling the images or specify how the upscaling is done. The typical 2x-bilinear-filtered upscaling, done by most browsers, is not always desirable. In addition, as display vendors pack more pixels in, what happens when the “high-resolution” version needed is 4x or 8x?
  • SVG/Vector-based images aren’t always the answer. There’s a lot of benefits to vector-based formats, but they’re not the holy grail many think they are. For vector-based images, rendering costs grow as you add details with polygons and paths. It’s why video games still rely heavily on texture mapping, even as graphics hardware has progressed to handle rendering millions of polygons per frame – the additional geometry and computation for fine details is enormous.

glfx – WebGL basis

The base code for my WebGL experiments have been pretty sloppy thus far. I recently took some time to cleanup the code in order to have a more solid basis to work from and I’m presenting it here as a primer for anyone looking for a simple bootstrap or a code-heavy intro to WebGL.

A walk-through of the base code (glfx) and sample code to generate the demo shown below follows. The code is also available via the glfx bitbucket repository.


For matrix and vector operations, the glMatrix library.

Also window.requestAnimationFrame needs to be defined. For older browsers the following shim can be used:

window.requestAnimationFrame = (function(time){
return window.requestAnimationFrame ||
         window.webkitRequestAnimationFrame ||
         window.mozRequestAnimationFrame ||
         window.oRequestAnimationFrame ||
         window.msRequestAnimationFrame ||
function( callback ){
            window.setTimeout(callback, 1000 / 60);


glfx is the crux of the rendering interface and encapsulates the WebGL context, functionality to load assets (shaders, textures, models), and functionality to setup and render the scene.

// glfx object wraps everything necessary for the rendering interface
var glfx = { };

// echo function to output debug statements to console
glfx.echo = function(txt) {
if(typeof console.log !== 'undefined') {

// WebGL context = null;

// reference count for assets needed before rendering()
glfx.assetRef = 0;
// function to call when all assets are loaded, set by user via glfx.whenAssetsLoaded, reset internally
glfx.onAssetsLoaded = function() { };
// function to schedule callback when all assets are loaded, set by user
glfx.whenAssetsLoaded = function(_callback) {
if(typeof _callback !== 'undefined') {
if(glfx.assetRef === 0) {
else {
glfx.onAssetsLoaded = _callback;
// function to increment asset ref count
glfx.incAssetRef = function() {
if(glfx.assetRef === 0) {
glfx.onAssetsLoaded =
function() { }; // reset
// function to decrement asset ref count
glfx.decAssetRef = function() {

// Shaders class
glfx.shaders = { };
// buffer to store loaded shaders
glfx.shaders.buffer = new Array();

// Function to load vertex shader from external file
// _url = path to shader source
// _callback = function to call after shader is created, shader object passed is shader is successfully compiled, null otherwise
glfx.shaders.load = function(_url, _name, _type, _callback) {

var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange =
function() {                
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {

var shaderSrc = xmlhttp.responseText;
var shader =;, shaderSrc);;

if (!, {
shader =

if(typeof _callback !== 'undefined') {

glfx.shaders.buffer[_name] = shader;

"GET", _url, true);

// Textures class
glfx.textures = { };
// Textures array
glfx.textures.buffer = new Array();
// Method to load texture from file
glfx.textures.load = function(_path, _name) {


glfx.textures.buffer[_name] =;

var tex=glfx.textures.buffer[_name];
tex.image =
new Image();
tex.image.onload =
function() {                

var tex = glfx.textures.buffer[_name];                                                   , tex);,
true);, 0,,,, tex.image);,,;,,;

// required for non-power-of-2 textures,,;,,;,



tex.image.src = _path;            

// Model class
glfx.model = function() {

this.vertexBuffer = null;
this.indexBuffer = null;
this.texcoordBuffer = null;
this.normalBuffer = null;


// Models class
glfx.models = { };
// Models array
glfx.models.buffer = new Array();
// Method to load models from JSON file
glfx.models.load = function(_url, _name, _callback) {


var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange =
function() {                
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {

var data = JSON.parse(xmlhttp.responseText);

var mdl = new glfx.model();

mdl.vertexBuffer =;, mdl.vertexBuffer);,
new Float32Array(data.verts),;
mdl.vertexBuffer.itemSize = 3;
mdl.vertexBuffer.numItems = data.verts.length / 3;

mdl.indexBuffer =;, mdl.indexBuffer);,
new Uint16Array(data.indices),;
mdl.indexBuffer.itemSize = 1;
mdl.indexBuffer.numItems = data.indices.length;        

if(data.texcoords.length > 0) {
mdl.texcoordBuffer =;, mdl.texcoordBuffer);,
new Float32Array(data.texcoords),;
mdl.texcoordBuffer.itemSize = 2;
mdl.texcoordBuffer.numItems = data.texcoords.length / 2;            

if(data.normals.length > 0) {
mdl.normalBuffer =;, mdl.normalBuffer);,
new Float32Array(data.normals),;
mdl.normalBuffer.itemSize = 3;
mdl.normalBuffer.numItems = data.normals / 3;

glfx.models.buffer[_name] = mdl;

"GET", _url, true);

// Scene class
glfx.scene = { };
// Scene last render time
glfx.scene.ptime = 0;
// Model-View matrix
glfx.scene.matModelView = null;
// Perspective matrix
glfx.scene.matPerspective = null;
// Scene graph
glfx.scene.graph = new Array();

// Class for scene (world) objects
// _base = object with vertex buffer, index buffer, texture coordinate buffer, etc.
glfx.scene.worldObject = function(_base, _shaderProgram) {
this.base = _base;            
this.shprog = _shaderProgram;
this.position = vec3.create();
this.rotation = vec3.create();
this.scale = vec3.create([1.0, 1.0, 1.0]);
this.update = function() { };

// method to add object to scene graph
glfx.scene.addWorldObject = function(_wo) {

// set field of view
glfx.setFOV = function(_fov) {
mat4.perspective(_fov, /, 0.1, 100.0, glfx.scene.matPerspective);

// set clear color
glfx.setClearColor = function(_color) {[0], _color[1], _color[2], _color[3]);

// Initialization function
// _canvas = DOM canvas element
// _onInitComplete (optional) = callback after init is complete
glfx.init = function(_canvas, _onInitComplete) { = _canvas.getContext(
"experimental-webgl", {antialias:true});
if (! {
"No webGL support.");
return false;

// Set viewport width,height based on dimensions of canvas element = _canvas.width; = _canvas.height;            

// Set clear color

// Enable depth buffer;                

// Setup scene matrices
glfx.scene.matPerspective = mat4.create();
glfx.scene.matModelView = mat4.create();            

// Reset render target, null);,

// Execute callback if one was passed
if(typeof _onInitComplete !== 'undefined') {

// Begin rendering

return true;

// Render loop function
glfx.render = function(time) {


if(glfx.assetRef < 0) {

// Reset framebuffer, null);        

// Clear viewport, 0,,; |;                    

// Calculate frame time delta
var tdelta = 0;
if(glfx.scene.ptime > 0) {
tdelta = time - glfx.scene.ptime;
glfx.scene.ptime = time;

// Render all models in scene
for(var i=0; i<glfx.scene.graph.length; i++) {


glfx.scene.graph[i].update(tdelta, glfx.scene.graph[i]);
var objpos = glfx.scene.graph[i].position;
var objrot = glfx.scene.graph[i].rotation;
var objscale = glfx.scene.graph[i].scale;

mat4.scale(glfx.scene.matModelView, objscale);
mat4.translate(glfx.scene.matModelView, objpos);
mat4.rotate(glfx.scene.matModelView, objrot[0], [1, 0, 0]);                
mat4.rotate(glfx.scene.matModelView, objrot[1], [0, 1, 0]);        
mat4.rotate(glfx.scene.matModelView, objrot[2], [0, 0, 1]);                        

glfx.scene.graph[i].render(tdelta, glfx.scene.graph[i], glfx.scene.matModelView, glfx.scene.matPerspective);


Initializing glfx

Initializing glfx simply involves calling the glfx.init() function with the canvas element that’s going to be used to render on.

var canvasElem = document.getElementById('wgl-canvas');

This will setup the rendering interface which will begin rendering frames, but as there is nothing in the scene only a clear is done when a frame is rendered. The clear color is set to white (1,1,1,1) and the field of view set to 90deg by default; these can be changed with the glfx.setClearColor() and glfx.setFOV() methods, respectively.

Loading assets

Assets (shaders, textures, and models) are loaded asynchronously via AJAX requests. As there may be dependencies on multiple assets for rendering and scene creation, a simple semaphore is used, glfx.assetRef.

  • glfx.assetRef is decremented when a new request for an asset is issued and incremented once the AJAX call succeeds and the asset has been created.
  • When glfx.assetRef < 0, it indicates a pending asset for the scene and no rendering is done.
  • A callback can be scheduled for when glfx.assetRef = 0 (i.e. all pending assets loaded) via the glfx.whenAssetsLoaded() method.
// Load basic shaders for rendering
glfx.shaders.load('basic.vs', "vert-shader-basic",;
'basictex.fs', "frag-shader-tex",;

// Load necessary textures
glfx.textures.load('img/test.png', 'test-tex');                    

// Load models used in scene
glfx.models.load('cube.json', 'cubemdl', glfx.models.jsonParser);

Note that all the asset load methods take a URL as the first argument, and a name as the second argument. The name is an identifier by which to lookup the asset from the buffer it’s stored in. Also, glfx.models.jsonParser is the only model parser available and loads models corresponding to the JSON data produced by my Wavefront OBJ to JSON converter.

Building a scene

After assets are loaded, we can can create shader programs and world objects, then add them to the scene.

glfx.whenAssetsLoaded(function() {

// Create shader program from loaded shaders
var shprog =;, glfx.shaders.buffer[
'vert-shader-basic']);, glfx.shaders.buffer[

if (!, {
"Could not create shader program");
return false;

// Setup variables for shader program
shprog.vertexPositionAttribute =, "aVertexPosition");;            

shprog.pMatrixUniform =,
shprog.mvMatrixUniform =,

shprog.textureCoordAttribute =,

// add some cubes to the scene graph
var cubeA = new glfx.scene.worldObject(glfx.models.buffer['cubemdl'], shprog);
cubeA.position = vec3.create([-1.6, 0.0, -25.0]);
cubeA.rotation = vec3.create([0.0, 0.0, 0.0]);
cubeA.scale = vec3.create([0.70, 1.0, 1.0]);
cubeA.render =
function(tdelta, wobj, matModelView, matPerspective) {
// Setup shader program to use
var shprog = wobj.shprog;;    

var tex = glfx.textures.buffer['test-tex'];       ;, tex);, 0);, wobj.base.vertexBuffer);, wobj.base.vertexBuffer.itemSize,,
false, 0, 0);        , wobj.base.texcoordBuffer);, wobj.base.texcoordBuffer.itemSize,,
false, 0, 0);            ,
false, matPerspective);,
false, matModelView);        , wobj.base.indexBuffer);, wobj.base.indexBuffer.numItems,, 0);    

cubeA.update =
function(tdelta, wobj) {

// some code to position and spin cubeA

if(wobj.position[2] < -5.0) {
wobj.position[2] += 0.022 * tdelta;
else {
wobj.position[2] = -5.0;

wobj.rotation[0] = 0.35;
wobj.rotation[1] += -(75 * tdelta) / 50000.0;
if( Math.abs(wobj.rotation[1]) >= 2.0*Math.PI ) {
wobj.rotation[1] = 0.0;
glfx.scene.addWorldObject( cubeA );

// Add another cube to the scene
var cubeB = new glfx.scene.worldObject(glfx.models.buffer['cubemdl'], shprog);
cubeB.position = vec3.create([1.6, 0.0, -25.0]);
cubeB.rotation = vec3.create([0.0, 0.0, 0.0]);
cubeB.scale = vec3.create([0.70, 1.0, 1.0]);
cubeB.update =
function(tdelta, wobj) {
// some code to position and spin cubeB
if(cubeA.position[2] > -15.0) {
if(wobj.position[2] < -5.0) {
wobj.position[2] += 0.022 * tdelta;
else {
wobj.position[2] = -5.0;

wobj.rotation[0] = 0.35;
wobj.rotation[1] += -(75 * tdelta) / 50000.0;
if( Math.abs(wobj.rotation[1]) >= 2.0*Math.PI ) {
wobj.rotation[1] = 0.0;

cubeB.render =
function(tdelta, wobj, matModelView, matPerspective) {
// Setup shader program to use
var shprog = wobj.shprog;;    

var tex = glfx.textures.buffer['test-tex'];       ;, tex);, 0);, wobj.base.vertexBuffer);, wobj.base.vertexBuffer.itemSize,,
false, 0, 0);        , wobj.base.texcoordBuffer);, wobj.base.texcoordBuffer.itemSize,,
false, 0, 0);            ,
false, matPerspective);,
false, matModelView);        , wobj.base.indexBuffer);, wobj.base.indexBuffer.numItems,, 0);    

glfx.scene.addWorldObject( cubeB );


Shaders for programs are pulled from the glfx.shaders.buffer[] associative array, referenced by the name specified when they were loaded.

Once we have a shader program and a model, we can create items for the scene by constructing glfx.scene.worldObject objects:

  • Construct the glfx.scene.worldObject object by specifying a model from the glfx.models.buffer[] associative array and the shader program as arguments to the constructor.
  • The worldObject.position, worldObject.rotation, and worldObject.scale vectors can be set as desired.
  • The worldObject.update() method can be overridden to describe how to manipulate the object in each frame.
  • The worldObject.render() method can be overridden to render the objects making use of the underlying buffers in worldObject.base: worldObject.base.indexBuffer, worldObject.base.vertexBuffer, worldObject.base.normalBuffer, worldObject.base.texcoordBuffer, as well as textures from the glfx.textures.buffer[] associative array.
  • Note that transformation on the model-view matrix (matModelView) is done within glfx.render() and should not be done within worldObject.render().

This callback is not ideal. I’m exposing a lot of rendering code that would best be abstracted away to glfx. However, without a strict definition of how a model should be textured or what variables are to be passed over to the vertex and fragment shaders, abstracting further is premature.