Archive for April, 2015

Vertical centering with flexbox

I’ve only begun delving into doing layouts with flexbox (CSS Flexible Box Layout), but one immediate use case is vertical centering, which has been a pain point in web design since the transition away from table-based layouts.

HTML

<body>    
<
div class="flex-container">    
    <
p class="centered-element-within-body">Center Me</p>            
</
div>
</
body>

CSS

* { margin:0; padding:0; border:none; box-sizing:border-box; }
html, body { width:100%; height:100%; overflow:hidden; }

.flex-container { width:100%; /* width of body (fluid) */
height:100%; /* height of body (fluid) */
display:flex; /* flex display */
flex-direction: row; /* flow of elements in container */
justify-content: center; /* alignment along the main axis (x-axis, since the flex-direction = row) */
                    
align-items: center; /* alignment along the secondary axis (y-axis) */            
                    
border: 5px solid #2C81F5;                    
                }
                
.centered-element-within-body { background:#2C81F5; color:#fff; }

Real-time image processing on the web

A while ago I began playing around with grabbing a video stream from a webcam and seeing what I could do with the captured data. Capturing the video stream using the navigator.getUserMedia() method was straightforward, but directly reading and writing the image data of the video stream isn’t possible. That said, the stream data can be put onto a canvas using CanvasRenderingContext2D.drawImage(), giving you to ability to read the pixel data. When it comes to writing visual data, a few options are available.

var videoElem = document.querySelector('video');

// Request video stream
navigator.getUserMedia({video: true, audio: false},

function(_localMediaStream) {
videoStream = _localMediaStream;
videoElem.src = window.URL.createObjectURL(_localMediaStream);
},

function(err) {
console.log(
'navigator.getUserMedia error' + err);
}

);
var videoElem = document.querySelector('video');
var canvas = document.querySelector('canvas');
var ctx = canvas.getContext('2d');

...

// put snapshot from video stream into canvas
ctx.drawImage(videoElem, 0, 0);

You can read and write to the <canvas> element, so hiding the <video> element with the source data and just showing the <canvas> element is an option, but the CanvasRenderingContext2D.drawImage() call is expensive; looking at the copied stream on the <canvas> element there is, very noticeable, visual lag. Another reason to avoid this option is that the frequency at which you render (e.g. 30 FPS), isn’t necessarily the frequency at which you’d want to grab and process image data (e.g. 10 FPS). The disassociation allow you to keep the video playback smooth, for a better user experience, but more effectively utilize CPU cycles for the image processing. At least in my experiences so far, a small delay in the visual feedback from the image processing is acceptable and looks perfectly fine intermixed with the higher-frequency video stream.

Throwing aside reading and writing to just the <canvas> element, alternative options all involve showing the <video> element with the webcam stream and placing visual feedback on top of the video pixels. A few ideas:

  • Write pixel data to another canvas and render it on top of the <video> element
  • Render SVG elements on top of the <video> element
  • Render DOM elements (absolutely positioned) on top of the <video> element

The third option is an ugly solution, but it’s fast to code and thus allows for quick prototyping. The demo and code below shows a quick demo I slapped together using <div> elements as markers for hot spots, in this case bright spots, within the video.

<!DOCTYPE html>
<
html>
<
head>
<
title>Webcam Cap</title>
<
meta charset="UTF-8">
<
meta name="viewport" content="width=device-width, initial-scale=1.0">

<
style type="text/css">
* { margin:0; padding:0; border:none; }
</style>

</
head>

<
body>
<
div>
<
video style="width:640px; height:480px;" width="640" height="480" autoplay></video>
<
canvas style="display:none; width:640px; height:480px;" width="640" height="480"></canvas>
</
div>

<
div class="ia-markers"></div>

<
script type="text/javascript">

navigator.getUserMedia = (navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia);

if ( typeof navigator.getUserMedia !== 'undefined' ) {

var videoElem = document.querySelector('video');
var canvas = document.querySelector('canvas');
var ctx = canvas.getContext('2d');
var videoStream = null;
var snapshotIntv = null;

var width = 640;
var height = 480;

// Request video stream
navigator.getUserMedia({video: true, audio: false},

function(_localMediaStream) {
videoStream = _localMediaStream;
videoElem.src = window.URL.createObjectURL(_localMediaStream);

// Take a snapshot of the video stream 10ms
snapshotIntv = setInterval(function() {
processSnapshot(videoStream);
}, 100);

},

function(err) {
console.log(
'navigator.getUserMedia error' + err);
}

);


// Take a snapshot from the video stream
function processSnapshot() {

// put snapshot from video stream into canvas
ctx.drawImage(videoElem, 0, 0);

// Clear old snapshot markers
var markerSetParent = (document.getElementsByClassName('ia-markers'))[0];
markerSetParent.innerHTML =
'';

// Array to store hotzone points
var hotzones = [];

// Process pixels
var imageData = ctx.getImageData(0, 0, width, height);
for (var y = 0; y < height; y+=16) {
for (var x = 0; x < width; x+=16) {
var index = (x + y * imageData.width) << 2;

var r = imageData.data[index + 0];
var g = imageData.data[index + 1];
var b = imageData.data[index + 2];

if(r > 200 && g > 200 && b > 200) {
hotzones.push([x,y]);
}
}
}

// Add new hotzone elements to DOM
for(var i=0; i<hotzones.length; i++) {
var x = hotzones[i][0];
var y = hotzones[i][1];

var markerDivElem = document.createElement("div");
markerDivElem.setAttribute(
'style', 'position:absolute; width:16px; height:16px; border-radius:8px; background:#0f0; opacity:0.25; left:' + x + 'px; top:' + y + 'px');
markerDivElem.className =
'ia-hotzone-marker';

markerSetParent.appendChild(markerDivElem);
}
}

}
else {
console.log(
'getUserMedia() is not supported in your browser');
}

</script>

</
body>
</
html>