Timeout your XHR requests

Client-side timeouts on XHR requests isn’t something I’ve ever thought a whole lot about. The default is no timeout and in most cases, where you’re kicking off an XHR request in response to a user interaction, you probably won’t ever notice an issue. That said, I ran into a case with ScratchGraph on Chrome where not having a timeout specified, along with some client-side network errors, left the application in a state where it was unable to send any more XHR requests.

ScratchGraph continuously polls its server for new data and every so often I would notice that the XHR calls would stop, with the application left in a broken state, unable to make any AJAX calls. This typically (but not always) occurred when the machine woke up from being put to sleep and in the console there would be a few error messages, typically a number of ERR_NETWORK_IO_SUSPENDED and ERR_INTERNET_DISCONNECTED errors. Testing within my development environment, it was impossible to reproduce. Finally, I came across this StackOverflow post that pointed out that not having a timeout specified on the XHR calls would result in these errors.

I’m still not exactly sure of the interplay between Chrome, the XHR requests, and the network state that results in this situation, but since adding a timeout, I’ve yet to notice this behavior again. It’s also worth noting that it’s very simple to add a timeout on an XHR request:

var xhr = new XMLHttpRequest();
xhr.open(
'GET', '/hello', true);
xhr.timeout = 500;
// time in milliseconds


Logging to stdout and a file

A simple way to log to both stdout and a file (using a pipe and tee):

./myapp 2>&1 | tee -a myapp.log


PostgreSQL database import with Ansible

I had a hard time pulling together all the steps needed to import a PostgreSQL database using Ansible. Here’s the Ansible YAML blocks used to import the seed database for Lexiio.

1. Install PostgreSQL - name: Install Postgres
apt: name={{ item }} update_cache=yes cache_valid_time=3600 state=present
sudo: yes
with_items:
- postgresql
- postgresql-contrib
- libpq-dev
- python-psycopg2
tags: packages

2. Create the database (lexiiodb), UTF-8 for encoding and collation

- name: Create lexiiodb database
sudo_user: postgres
postgresql_db: name=lexiiodb encoding='UTF-8' lc_collate='en_US.UTF-8' lc_ctype='en_US.UTF-8' state=present

3. Create a role that will be granted access to the database (password is a variable read from some secret source)

- name: Create lexiio role for database
sudo_user: postgres
postgresql_user: db=lexiiodb user=lexiio password="{{ password }}" priv=ALL state=present

4. Start the PostgreSQL service

- name: Start the Postgresql service
sudo: yes
service:
name: postgresql
state: started
enabled: true

5. Import data into the database (using psql to pull in data from /home/lexiiodb.dump.sql)

- name: Importing lexiiodb data
sudo_user: postgres
shell: psql lexiiodb < /home/lexiiodb.dump.sql

6. For the role created, grant permissions on all schemas in the DB

- name: Grant usage of schema to lexiio role
sudo_user: postgres
postgresql_privs: database=lexiiodb state=present privs=USAGE type=schema roles=lexiio objs=dictionary

7. For the role created, grant permissions on all tables in the DB

- name: Grant table permissions for lexiio role
sudo_user: postgres
postgresql_privs: database=lexiiodb schema=dictionary state=present privs=SELECT,INSERT,UPDATE type=table roles=lexiio grant_option=no objs=ALL_IN_SCHEMA

8. For the role created, grant permissions on all sequences in the DB

- name: Grant sequence permissions for lexiio role
sudo_user: postgres
postgresql_privs: database=lexiiodb schema=dictionary state=present privs=USAGE type=sequence roles=lexiio grant_option=no objs=ALL_IN_SCHEMA


A more relational dictionary

As I started looking to add more functionality to Lexiio, I realized the Wiktionary definitions database dump I was using wasn’t going to cut it; specifically, I needed a normalized schema, or I’d have data duplication all over the place. I started normalizing in MySQL, but whether it was MySQL or MySQL Workbench, I kept running into character encoding issues. Using a simple INSERT-SELECT, in MySQL 5.7, to transfer words from the existing table to a new table resulted losing characters:

MySQL losing characters

I dumped the data into PostgreSQL, didn’t encounter the issue, and just kept working from there.

The normalized schema can be downloaded here: LexiioDB normalized
(released under the Creative Commons Attribution-ShareAlike License)

LexiioDB schema

The unknown_words and unknown_to_similar_words tables is specific to Lexiio and serve as a place to store unknown words entered by the user and close/similar matches to known words (via the Levenshtein distance).


Post-process shaders in glfx

Pushed an update to glfx to allow for post-process shading. When a post-process shader is defined, the scene is rendered to a screen-space quad (the size of the viewport), and that quad is then rendered to the viewport with the post-process shader applied.

The shader is loaded (asynchronously) like any other:

glfx.shaders.load('screenspace.fs', "frag-shader-screenspace", glfx.gl.FRAGMENT_SHADER);

Once loaded, we create the shader program, and get locations for whatever variables are used. The vertex shader isn’t anything special, it just transforms a vertex by the model-view and projection matrices, and passes along the texture coordinates.

glfx.whenAssetsLoaded(function() {

var postProcessShaderProgram = glfx.shaders.createProgram([glfx.shaders.buffer['vert-shader-basic'], glfx.shaders.buffer['frag-shader-screenspace']],
function(_shprog) {

// Setup variables for shader program
_shprog.vertexPositionAttribute = glfx.gl.getAttribLocation(_shprog, "aVertexPosition");
_shprog.pMatrixUniform = glfx.gl.getUniformLocation(_shprog,
"uPMatrix");
_shprog.mvMatrixUniform = glfx.gl.getUniformLocation(_shprog,
"uMVMatrix");
_shprog.textureCoordAttribute = glfx.gl.getAttribLocation(_shprog,
"aTextureCoord");

_shprog.uPeriod = glfx.gl.getUniformLocation(_shprog,
"uPeriod");
_shprog.uSceneWidth = glfx.gl.getUniformLocation(_shprog,
"uSceneWidth");
_shprog.uSceneHeight = glfx.gl.getUniformLocation(_shprog,
"uSceneHeight");

glfx.gl.enableVertexAttribArray(_shprog.vertexPositionAttribute);
glfx.gl.enableVertexAttribArray(_shprog.textureCoordAttribute);

});

...

We then tell glfx to apply our post-process shader program:

glfx.scene.setPostProcessShaderProgram(postProcessShaderProgram);

This call will result in different rendering path, which renders the scene to a texture, applies that texture to a screen-space quad, and renders the quad with the post-process shader.

Here is the shader for screenspace.fs, used in the demo shown above:

precision mediump float;

uniform float uPeriod;
uniform float uSceneWidth;
uniform float uSceneHeight;
uniform sampler2D uSampler;        
varying vec2 vTextureCoord;

void main(void) {

vec4 sum = vec4( 0. );
float blurSampleOffsetScale = 2.8;
float px = (1.0 / uSceneWidth) * blurSampleOffsetScale;
float py = (1.0 / uSceneHeight) * blurSampleOffsetScale;

vec4 src = texture2D( uSampler, ( vTextureCoord ) );

sum += texture2D( uSampler, ( vTextureCoord + vec2(-px, 0) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(-px, -py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(0, -py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(px, -py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(px, 0) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(px, py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(0, py) ) );
sum += texture2D( uSampler, ( vTextureCoord + vec2(-px, py) ) );
sum += src;

sum = sum / 9.0;

gl_FragColor = src + (sum * 2.5 * uPeriod);

}

Note that it requires a few uniforms to be supplied to it, we use the glfx.scene.onPostProcessPreDraw() callback to setup the variables (before the post-processed scene is drawn):

var timeAcc = 0;
glfx.scene.onPostProcessPreDraw =
function(tdelta) {

timeAcc += tdelta;
var timeScaled = timeAcc * 0.00107;

if(timeScaled > 2.0*Math.PI) {
timeScaled = 0;
timeAcc = 0;
}

var period = Math.cos(timeScaled);
glfx.gl.uniform1f(postProcessShaderProgram.uPeriod, period + 1.0);

glfx.gl.uniform1f(postProcessShaderProgram.uSceneWidth, glfx.gl.viewportWidth);
glfx.gl.uniform1f(postProcessShaderProgram.uSceneHeight, glfx.gl.viewportHeight);
};

What we’re doing is using the scene rendering time deltas to generate a periodic/sinusoidal wave. This results in the pulsing brightness/fading effect of the scene. The brightness effect itself is done by adding the source pixel to a blurred + brightened version of itself. The blurring allows for the soft fade in and fade out.


Lexiio

Another little experiment of mine: Lexiio, a web-based CLI dictionary.

Lexiio

A few takeaways:

  • Part of the reason for building this was that I wanted to actually make use of the Wiktionary data set snapshot in a real project. The data set is pretty comprehensive, and easy to parse and work with.
  • This was also a learning exercise for Golang. There’s nothing complex here but, so far, working with Go has been enjoyable. I like that I’m building a native application, types are enforced, and the HTTP server included as part of the standard library is incredibly easy to setup and work with.
  • I wanted to experiment a bit with what a web-based CLI would look and feel like. For something like a dictionary, where user interaction revolves around textual input/output, a command-line interface seems to work really well.


The end of XULRunner

I’ve written a bit on XULRunner (and the related technologies, XUL and XPCOM) for application development, so I felt it was apt to mention that Mozilla no longer provides new builds of XULRunner. The recommended way to run XULRunner-based application is via the Firefox executable (Firefox ships with a private private XULRunner package):

firefox -app application.ini

You can take a copy of a Firefox installation and or compile Firefox from source, which is surprisingly easy.

One nice side effect worth mentioning: while I could never get proper high-DPI support with XULRunner, it works perfectly when Firefox is used as the runner.


GLSL variable qualifiers

I’ve been playing around with WebGL shader code recently and found this bit on variable prefixes helpful, particularly in the explanation of the variable qualifiers:

  • Attribute: data provided by buffers
  • Uniform: inputs to the shaders
  • Varying: values passed from a vertex shader to a fragment shader and interpolated (or varied) between the vertices for each pixel drawn

Something important to keep in mind is that this relates to the OpenGL ES Shading Language, Version 1.00, which is (unfortunately) what’s currently supported by WebGL.

A WebGL implementation must only accept shaders which conform to The OpenGL ES Shading Language, Version 1.00 [GLES20GLSL], and which do not exceed the minimum functionality mandated in Sections 4 and 5 of Appendix A.

Attribute and Varying were part of early versions of, OpenGL-supported, GLSL, but are deprecated as of OpenGL 3.0 / GLSL 1.30.10, and replaced with more generic constructs:

  • in is for input from the previous pipeline stage, i.e. per vertex (or per fragment) values at most, per primitive if using glAttribDivisor and hardware instanciation
  • out is for output to the next stage


Vertical centering with flexbox

I’ve only begun delving into doing layouts with flexbox (CSS Flexible Box Layout), but one immediate use case is vertical centering, which has been a pain point in web design since the transition away from table-based layouts.

HTML

<body>    
<
div class="flex-container">    
    <
p class="centered-element-within-body">Center Me</p>            
</
div>
</
body>

CSS

* { margin:0; padding:0; border:none; box-sizing:border-box; }
html, body { width:100%; height:100%; overflow:hidden; }

.flex-container { width:100%; /* width of body (fluid) */
height:100%; /* height of body (fluid) */
display:flex; /* flex display */
flex-direction: row; /* flow of elements in container */
justify-content: center; /* alignment along the main axis (x-axis, since the flex-direction = row) */
                    
align-items: center; /* alignment along the secondary axis (y-axis) */            
                    
border: 5px solid #2C81F5;                    
                }
                
.centered-element-within-body { background:#2C81F5; color:#fff; }


Real-time image processing on the web

A while ago I began playing around with grabbing a video stream from a webcam and seeing what I could do with the captured data. Capturing the video stream using the navigator.getUserMedia() method was straightforward, but directly reading and writing the image data of the video stream isn’t possible. That said, the stream data can be put onto a canvas using CanvasRenderingContext2D.drawImage(), giving you to ability to read the pixel data. When it comes to writing visual data, a few options are available.

var videoElem = document.querySelector('video');

// Request video stream
navigator.getUserMedia({video: true, audio: false},

function(_localMediaStream) {
videoStream = _localMediaStream;
videoElem.src = window.URL.createObjectURL(_localMediaStream);
},

function(err) {
console.log(
'navigator.getUserMedia error' + err);
}

);
var videoElem = document.querySelector('video');
var canvas = document.querySelector('canvas');
var ctx = canvas.getContext('2d');

...

// put snapshot from video stream into canvas
ctx.drawImage(videoElem, 0, 0);

You can read and write to the <canvas> element, so hiding the <video> element with the source data and just showing the <canvas> element is an option, but the CanvasRenderingContext2D.drawImage() call is expensive; looking at the copied stream on the <canvas> element there is, very noticeable, visual lag. Another reason to avoid this option is that the frequency at which you render (e.g. 30 FPS), isn’t necessarily the frequency at which you’d want to grab and process image data (e.g. 10 FPS). The disassociation allow you to keep the video playback smooth, for a better user experience, but more effectively utilize CPU cycles for the image processing. At least in my experiences so far, a small delay in the visual feedback from the image processing is acceptable and looks perfectly fine intermixed with the higher-frequency video stream.

Throwing aside reading and writing to just the <canvas> element, alternative options all involve showing the <video> element with the webcam stream and placing visual feedback on top of the video pixels. A few ideas:

  • Write pixel data to another canvas and render it on top of the <video> element
  • Render SVG elements on top of the <video> element
  • Render DOM elements (absolutely positioned) on top of the <video> element

The third option is an ugly solution, but it’s fast to code and thus allows for quick prototyping. The demo and code below shows a quick demo I slapped together using <div> elements as markers for hot spots, in this case bright spots, within the video.

<!DOCTYPE html>
<
html>
<
head>
<
title>Webcam Cap</title>
<
meta charset="UTF-8">
<
meta name="viewport" content="width=device-width, initial-scale=1.0">

<
style type="text/css">
* { margin:0; padding:0; border:none; }
</style>

</
head>

<
body>
<
div>
<
video style="width:640px; height:480px;" width="640" height="480" autoplay></video>
<
canvas style="display:none; width:640px; height:480px;" width="640" height="480"></canvas>
</
div>

<
div class="ia-markers"></div>

<
script type="text/javascript">

navigator.getUserMedia = (navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia);

if ( typeof navigator.getUserMedia !== 'undefined' ) {

var videoElem = document.querySelector('video');
var canvas = document.querySelector('canvas');
var ctx = canvas.getContext('2d');
var videoStream = null;
var snapshotIntv = null;

var width = 640;
var height = 480;

// Request video stream
navigator.getUserMedia({video: true, audio: false},

function(_localMediaStream) {
videoStream = _localMediaStream;
videoElem.src = window.URL.createObjectURL(_localMediaStream);

// Take a snapshot of the video stream 10ms
snapshotIntv = setInterval(function() {
processSnapshot(videoStream);
}, 100);

},

function(err) {
console.log(
'navigator.getUserMedia error' + err);
}

);


// Take a snapshot from the video stream
function processSnapshot() {

// put snapshot from video stream into canvas
ctx.drawImage(videoElem, 0, 0);

// Clear old snapshot markers
var markerSetParent = (document.getElementsByClassName('ia-markers'))[0];
markerSetParent.innerHTML =
'';

// Array to store hotzone points
var hotzones = [];

// Process pixels
var imageData = ctx.getImageData(0, 0, width, height);
for (var y = 0; y < height; y+=16) {
for (var x = 0; x < width; x+=16) {
var index = (x + y * imageData.width) << 2;

var r = imageData.data[index + 0];
var g = imageData.data[index + 1];
var b = imageData.data[index + 2];

if(r > 200 && g > 200 && b > 200) {
hotzones.push([x,y]);
}
}
}

// Add new hotzone elements to DOM
for(var i=0; i<hotzones.length; i++) {
var x = hotzones[i][0];
var y = hotzones[i][1];

var markerDivElem = document.createElement("div");
markerDivElem.setAttribute(
'style', 'position:absolute; width:16px; height:16px; border-radius:8px; background:#0f0; opacity:0.25; left:' + x + 'px; top:' + y + 'px');
markerDivElem.className =
'ia-hotzone-marker';

markerSetParent.appendChild(markerDivElem);
}
}

}
else {
console.log(
'getUserMedia() is not supported in your browser');
}

</script>

</
body>
</
html>