Posts Tagged ‘ajax’

Rendering HTML to images with SVG foreignObject

Motivation

For applications that allow users to create visual content, being able to generate images of their work can be important in a number of scenarios: preview/opengraph images, allowing users to display content elsewhere, etc. This popped up as a need for ScratchGraph and led me to research a few possible solutions. Using the SVG <foreignObject> element was one of the more interesting solutions I came across, as all rendering and image creation is done client-side.

<foreignObject> to Image

<foreignObject> is a somewhat strange element. Essentially, it allows you to load and render arbitrary HTML content within SVG. This in and of itself isn’t helpful for generating an image, but we can take advantage of two other aspects of modern browsers to make this a reality:

  • SVG markup can be dynamically loaded into an Image by transforming the markup into a data URL
  • Data URL length limits are no longer a concern. We no longer have the kilobyte-scale limits we were dealing with a few years ago

Sketching it out, the process looks something like this (contentHtml is a string with the HTML content we want to render):

The code for this is pretty straightforward:

// build SVG string
const svg = `
<svg xmlns='http://www.w3.org/2000/svg' width='
${width}' height='${height}'>
<foreignObject x='0' y='0' width='
${width}' height='${height}'>
${contentHtml}
</foreignObject>
</svg>`
;

// convert SVG to data-uri
const dataUri = `data:image/svg+xml;base64,${window.btoa(svg)}`;

Here I’m assuming contentHtml is valid and can be trusted. If that’s not the case, you’ll likely need some pre-processing steps before sticking it into a string like this.

The code above works, to a degree; there’s a few key limitations to be aware of:

  • Cross-origin images served without CORS headers won’t load within <foreignObject>
  • Styles declared via stylesheets do not pass through to the contents of <foreignObject>
  • External resources (images, fonts, etc.) won’t be in the generated Image, as the browser doesn’t wait for these resources to be loaded before rendering out the image

The cross-origin issue may be annoying and unexpected (as the browser does load these images), but it’s a valid security measure and CORS provides the mechanism around it.

Handling stylesheets and external resources are more important concerns, and addressing them allows for a much more robust process.

Handling stylesheets

This isn’t anything too fancy, here are the steps involved:

  • Copy all the style rules, from all the stylesheets, in the parent document
  • Wrap all those rules in a <style> tag
  • Prepend that string to the contentHtml string

The code for this precursor step looks something like this:

const styleSheets = document.styleSheets;
let cssStyles = "";
let urlsFoundInCss = [];

for (let i=0; i<styleSheets.length; i++) {
for(let j=0; j<styleSheets[i].cssRules.length; j++) {
const cssRuleStr = styleSheets[i].cssRules[j].cssText;
cssStyles += cssRuleStr;
}
}

const styleElem = document.createElement("style");
styleElem.innerHTML = cssStyles;
const styleElemString = new XMLSerializer().serializeToString(styleElem);

...

contentHtml = styleElemString + contentHtml;

...

Handling external resources

My solution here is somewhat curd, but it’s functional.

  • Find url values in the CSS code or src attribute values in the HTML code
  • Make XHR requests to get these resources
  • Encode the resources as Base64 and construct data URLs
  • Replace the original URLs (in the CSS url or HTML src) with the new base64 data URLs

The following shows how this is done for the HTML markup (the process is only slightly different for CSS).

const escapeRegExp = function(string) {
return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
};

let urlsFoundInHtml = getImageUrlsFromFromHtml(contentHtml);
const fetchedResources = await getMultipleResourcesAsBase64(urlsFoundInHtml);
for(let i=0; i<fetchedResources.length; i++) {
const r = fetchedResources[i];
contentHtml = contentHtml.replace(
new RegExp(escapeRegExp(r.resourceUrl),"g"), r.resourceBase64);
}

The getImageUrlsFromFromHtml() and parseValue() methods that extract the value of src attributes from elements:

/**
*
*
@param {String} str
*
@param {Number} startIndex
*
@param {String} prefixToken
*
@param {String[]} suffixTokens
*
*
@returns {String|null}
*/
const parseValue = function(str, startIndex, prefixToken, suffixTokens) {
const idx = str.indexOf(prefixToken, startIndex);
if(idx === -1) {
return null;
}

let val = '';
for(let i=idx+prefixToken.length; i<str.length; i++) {
if(suffixTokens.indexOf(str[i]) !== -1) {
break;
}

val += str[i];
}

return {
"foundAtIndex": idx,
"value": val
}
};

/**
*
*
@param {String} str
*
@returns {String}
*/
const removeQuotes = function(str) {
return str.replace(/["']/g, "");
};

/**
*
*
@param {String} html
*
@returns {String[]}
*/
const getImageUrlsFromFromHtml = function(html) {
const urlsFound = [];
let searchStartIndex = 0;

while(true) {
const url = parseValue(html, searchStartIndex, 'src=', [' ', '>', '\t']);
if(url === null) {
break;
}

searchStartIndex = url.foundAtIndex + url.value.length;
urlsFound.push(removeQuotes(url.value));
}

return urlsFound;
};

The getMultipleResourcesAsBase64() and getResourceAsBase64() methods responsible for fetching resources:

/**
*
*
@param {String} url
*
@returns {Promise}
*/
const getResourceAsBase64 = function(url) {
return new Promise(function(resolve, reject) {
const xhr = new XMLHttpRequest();
xhr.open(
"GET", url);
xhr.responseType =
'blob';

xhr.onreadystatechange =
async function() {
if(xhr.readyState === 4 && xhr.status === 200) {
const resBase64 = await binaryStringToBase64(xhr.response);
resolve(
{
"resourceUrl": url,
"resourceBase64": resBase64
}
);
}
};

xhr.send(
null);
});
};

/**
*
*
@param {String[]} urls
*
@returns {Promise}
*/
const getMultipleResourcesAsBase64 = function(urls) {
const promises = [];
for(let i=0; i<urls.length; i++) {
promises.push( getResourceAsBase64(urls[i]) );
}
return Promise.all(promises);
};

More code

The code for this experiment is up on Github. Most functionality is encapsulated with the ForeignHtmlRenderer method, which contains the code shown in this post.

Other Approaches

  • Similar (same?) approach with dom-to-image
    This library also uses the <foreignObject> element and an approach similar to what I described in this post. I played around with it briefly and remember running to a few issues, but I didn’t keep the test code around and don’t remember what the errors were.
  • Server-side/headless rendering with puppeteer
    This seems to be the defacto solution and, honestly, it’s a pretty good solution. It’s not too difficult to get it up and running as a service, though there will be an infrastructure cost. Also, I’d be willing to bet this is what services like URL2PNG use on their backend.
  • Client-side rendering with html2canvas
    This is a really cool project that will actually parse the DOM tree + CSS and render the page (it’s a rendering engine done in client-side javascript). Unfortunately, only a subset of CSS is supported and SVG is not supported.

Timeout your XHR requests

Client-side timeouts on XHR requests isn’t something I’ve ever thought a whole lot about. The default is no timeout and in most cases, where you’re kicking off an XHR request in response to a user interaction, you probably won’t ever notice an issue. That said, I ran into a case with ScratchGraph on Chrome where not having a timeout specified, along with some client-side network errors, left the application in a state where it was unable to send any more XHR requests.

ScratchGraph continuously polls its server for new data and every so often I would notice that the XHR calls would stop, with the application left in a broken state, unable to make any AJAX calls. This typically (but not always) occurred when the machine woke up from being put to sleep and in the console there would be a few error messages, typically a number of ERR_NETWORK_IO_SUSPENDED and ERR_INTERNET_DISCONNECTED errors. Testing within my development environment, it was impossible to reproduce. Finally, I came across this StackOverflow post that pointed out that not having a timeout specified on the XHR calls would result in these errors.

I’m still not exactly sure of the interplay between Chrome, the XHR requests, and the network state that results in this situation, but since adding a timeout, I’ve yet to notice this behavior again. It’s also worth noting that it’s very simple to add a timeout on an XHR request:

var xhr = new XMLHttpRequest();
xhr.open(
'GET', '/hello', true);
xhr.timeout = 500;
// time in milliseconds

jxNotify

I made a little JavaScript notification system, somewhat inspired by webOS and also by the type of notifications you see on Gmail.

jxNotify code

The central idea was to have an elegant system that could sensibly display the progress of AJAX operations; meaning notifications stays up while the operation is being done (i.e. while sending request and waiting for a reply from the server), then a success or failure message is posted upon completion, which fades away automatically.

While designed for AJAX calls, this could certainly be used in other cases as well.

jxNotify notifyPre

jxNotify notifyPostError


Initializing

// optional argument = icon, recommended size of 18x18
jxNotify.init('jx-notify.content/sdotspott-notify-icon.png');


Notify of operation in progress (notifyPre)

jxNotify.notifyPre('doing stuff...');


Notify of operation completed (notifyPost)

jxNotify.notifyPost('finished!');


Notify of operation failure (notifyPostError)

jxNotify.notifyPostError('something bad happened!');

 

Reflex Feedback widget

I worked on a small AJAX widget for user feedback built atop jQuery UI: Reflex Feedback. It’s inspired by the widgets you see from services like Get Satisfaction and UserVoice, but much simpler and it’s a frontend-only widget, how you handle the feedback info on the backend is up to you.

Here’s what it looks like.

reflex feedback widget dialog

And here’s what the tag that opens the dialog looks like:

reflex feedback widget tag

To use it, download or clone the ReflexFeedback repo from bitbucket

Place the .js file wherever you’d like but the /reflex.content folder should a subdirectory in the same folder as the page loading the .js file. Load reflex.js as you would any other javascript file:

<script type="text/javascript" src="js/reflex.js"></script>

Call Reflex.init() to add the widget to the page. The first argument is the DOM element to attach the additional HTML/CSS code to. The seconds argument is the server-side script to call when the user clicks Send Feedback.

Reflex.init($('body'), 'controller/post_feedback.php');

That’s it for the frontend. You should see the tag show up in the right-hand corner and when clicked the dialog open.

For the backend, the AJAX call to send the feedback info will send a POST request with 2 fields: feedback_type, feedback_txt.

Reflex expects an XML reply from the server:

<reflex>
<result>ok</result>
</reflex>

ok indicates a successful result, any other reply is considered an error.

A successful result will close the dialog and show another with a thank you message.

reflex feedback thank you dialog

For an error, a message is shown below the Send Feedback button, informing the user that an error has occurred and to try again.

reflex feedback send fail

As for what to actually do with the feedback, that’s up to you, but what I’m doing is sending myself an email with the feedback info. I’ve posted my PHP script below; feel free to use it, modify it, etc. If you do use this code, be sure to fill in your mail server credentials and a from address; you’ll also need PEAR’s Mail package installed.

<?php

require_once "Mail.php";
require_once "Mail/mime.php";

header('Content-type: application/xml; charset=utf-8');
echo "<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n";

if(!isset($_POST['feedback_type']) || !isset($_POST['feedback_txt']))
{
echo "<reflex><result>error:missing-arguments</result></reflex>";
}
else
{
$from = "...";
$to = "...";
$subject = "Feedback from user...";

$feedback_type = $_POST['feedback_type'];
$feedback_txt = $_POST['feedback_txt'];

$bodyHtml = "<html><body>";
$bodyHtml .= "<p>Type: {$feedback_type}</p>";
$bodyHtml .= "<p>Feedback: {$feedback_txt}</p>";
$bodyHtml .= "</body></html>";
$body = $bodyHtml;

$host = "...";
$port = "...";
$username = "...";
$password = "...";

$headers = array('MIME-Version' => '1.0rn',
'Content-type' => 'text/html; charset=utf-8',
'From' => $from, 'To' => $to, 'Subject' => $subject);


$smtp = Mail::factory('smtp',
array ('host' => $host,
'port' => $port,
'auth' => true,
'username' => $username,
'password' => $password));

$mail = $smtp->send($to, $headers, $body);

if (PEAR::isError($mail))
{
$err_details = $mail->getMessage();
echo "<reflex><result>error:send-failure</result><details>{$err_details}</details></reflex>";
}
else
{
echo "<reflex><result>ok</result></reflex>";
}
}

?>

That’s all for now. I’ll work on more features and options for customization in the future. You can see the widget in action over at dotspott.com

progTools and Adobe Air

I made a little app to get my feet wet with Adobe Air. progTools just packages together a few common functions I find myself using frequently. You can get it my clicking the install badge (one of the very cool aspects of Adobe Air) below.

[airbadge]progTools, http://aautar.digital-radiation.com/apps/progTools.air,1.2,http://aautar.digital-radiation.com/progTools-air-badge/logo-badge.png[/airbadge]

(h/t to Peter Elst for the AIR Badge WordPress plugin)

What’s offered:

  • Conversion to/from a Unix timestamp
  • MD5 hash on a string
  • MD5 hash on a file
  • SHA1 hash on a string

progTools 1.2

Not too impressive, and only the MD5 file hash really utilizes a desktop feature of the Air framework, but it is somewhat useful and, at least in my case, I won’t end up going to Paj’s Home to use the javascript md5 implementation demo quite as often. Note, Paj’s MD5 library was used and I slightly modified core_md5() for the file hash to deal with hashing successive blocks. I’ll post the code soon.

I initially dismissed Air, back when it was Apollo, as I didn’t see the value in having yet another proprietary framework which didn’t really offer much beyond what was capable within a browser, aside from local file access. A few additions to the framework and a few realizations on my part have shifted my views:

  • Air supports HTML/CSS for layout and styling. Looking into cross-platform GUI frameworks, I’ve played around with WinForms (cross platform with Mono), Qt, Gtk, and wxWidgets. I’ve been disappointed to various degrees with all of them. It hit me that the most flexible and powerful cross-platform layout and styling framework out there is the HTML/CSS combo. It’s not perfect (e.g. floats, vertical centering) but it’s pretty damn good.
  • Support in Air 2 for sockets and interaction with native applications. This vastly opens the field for the types of applications possible with Air.
  • Market support from Adobe. The Air Marketplace is perhaps not too impressive, but it’s a major step in the right direction for desktop apps. Both Microsoft and Apple have their own stores planned, but with the success of such catalogs on smartphones for years now, why did it take so long to figure it out?
  • Install badges. They’re cool and important as they provide a bridge between the web and the desktop. Odd, but it seems Adobe more-so than Microsoft or Apple seems to understand the web-desktop relationship. Again, why is Adobe, a company that was fairly divorced from the desktop application space, the first to figure out that this was something important or at least the first to actually build it.

Now it’s not all sunshine and roses. Making an HTML/AJAX app in Air brings up a problem every AJAX developer has likely faced at some point. Javascript is slow… very slow. JavaScriptCore/Nitro, V8, Chakra, Tracemonkey… it doesn’t really matter (though performance improvements are being made), once your volume of data grows you’ll cringe at how slow things become. Coming from C++, C#, or even PHP, it’s painful to witness. In progTools a file only a few megabytes large will noticeably stall the application (I didn’t do the call asynchronously, but that’s besides the point). ActionScript is perhaps better and interop to a native executable could also alleviate the issue, but ultimately I’d simply like a faster JavaScript engine.

A second issue, relevant but not specific to Adobe Air, is code signing; you’ll notice the scary warning when installing progTools. Code signing is bullshit. Expensive bullshit. Yet, every platform developer is requiring it due to some misguided attempt at security. If you want to install progTools, the chain of trust is between me » this web server » you. Sticking a certificate authority in this chain is nonsense – a typical user will not know the CA and cannot establish any level of trust with some random, corporate CA.

Coding signing simply punishes small developers and establishes a new industry to leech from our wallets. In addition, as this user on StackOverflow asserts, it may well hamper the success of Air:

When you visit a site that lets you download an AIR app, it pops up big red screaming warnings about the imminent trashing of your computer, the theft of your identity and a life of torment[1]. Unless, of course, all the bedroom programmers decide to cough up the ongoing cost of certification.

User encouragement FAIL. Hobby developer encouragement FAIL. Technophobe terrorficiation avoidance FAIL.

I love AIR, but I don’t know what they were thinking with the installer. Laywers’ office moved closer to the developers’ over at HQ or something?

Anyways, I’m done ranting. I’ll eventually suck it up and get a certificate as I’m powerless to do anything else.

As for Air, I’ve just scratched the surface, but I’m impressed.

oh, and if you’d like to see something added to progTools, just let me know.

Safari 5 form submission bug

This applies specifically to Safari 5 (no problems in Safari 4) and forms with an enctype of multipart/form-data, submitted with a POST request built manually (for an AJAX [XMLHttpRequest] call).

Safari 5 injects a “charset=UTF-8” name/value pair into the ContentType field after the boundary string. This generates an invalid request. On the server-side of things, this resulted in no data in PHP’s $_POST global variable.

I tracked down the issues thanks to this post. The bug itself is a WebKit bug, described here.

The simple fix is to prevent Safari from automatically putting the charset parameter in by putting it in manually – before the boundary string. Here’s some jQuery code that demonstrates the issue/fix:

function ajaxSubmitForm(servlet, theForm, extraFunc)
{
var boundaryString = 'AaB03x';
var boundary = '--' + boundaryString;

var requestBody = new Array();

requestBody.push(boundary);

for (var i=0; i<theForm.length; i++)
{
requestBody.push('Content-Disposition: form-data; name="' + theForm.elements[i].name + '"');
requestBody.push('');
requestBody.push(theForm.elements[i].value);
requestBody.push(boundary);
}

var reqBodyText = requestBody.join('\r\n');

$.ajax({
url: servlet,
type: 'POST',
contentType: 'multipart/form-data; charset=UTF-8; boundary=' + boundaryString + '',
data: reqBodyText,
error: function(reqobj, errType, exceptionObj){
alert('Failed to submit form data.\r\nStatus = ' + reqobj.statusText);
},

success: function(xml)
{
if(extraFunc != null)
extraFunc();
}

});
}

XML DOM object to string conversion

One of those simple things which has eluded me for far too long. I finally found the solution here:

var string = (new XMLSerializer()).serializeToString(xmlobject); alert(string);

While the above is good to know, I’ve also discovered I don’t need it. I was looking to convert an XML DOM object, from an AJAX call, to plain text, but the XMLHttpRequest.responseText property provides just that. For jQuery users like myself, the XMLHttpRequest can be access via a synchronous jQuery.ajax call as the return value, or asynchronously via. the jQuery.ajax.complete callback or the jQuery.ajax.success callback (as of jQuery 1.4 the XMLHttpRequest object is now the third parameter).