Archive for March, 2011

Unity

I did a little experiment with meshing XML + HTML + PHP.

First, a little background info. Most service-side web technologies (PHP, Ruby on Rails, etc.) were designed around the concept of applying server-side markup to HTML,

<title><?php echo $title;?></title>

… which is interpreted and processed when the page is pulled. There are various frameworks and templating systems to make this easier, cleaner, more structured, etc. but the concept is the same. AJAX comes into play here and there, but it’s seen mostly as a way to sprinkle bits of dynamic data across a page.

This model was fine a few years ago, but for modern web services and apps I think there needs to be a push away from this model, primarily for 2 reasons:

  • Large portions, if not most, of the content of many web apps is dynamic, AJAX-queried data.
  • Support for mobile and desktop platforms connecting to web services is becoming increasingly important; platforms/apps for which HTML/CSS (or at least HTML/CSS designed for a browser) may not serve to provide the best UI.

AJAX structured code fits the bill.

A 100% AJAX architecture elegantly separates application logic (output as XML) from presentation (output as HTML/CSS). Unfortunately, 100% AJAX is not that easy for a complex application. JavaScript interpretation can be slow, killing frontend performance, and backend performance can suffer from the repeated flood of asynchronous requests. In addition, the latency from a multitude of AJAX requests+responses can quickly become noticeable (especially with a burdened backend and during an initial page load). When such issues become apparent, the solution always seems to be to go back to the messy world of hacking server-side tags into HTML.

Unity is an attempt to maintain a 100% AJAX application framework, but alleviate the issues mentioned above for web apps. It’s very, very simple; just a bit of glue that takes the XML from one or more AJAX calls (on localhost; the application server itself), turns it into JSON, and embeds it within the <head> element of the HTML document being served. This should not be done for every call, but for AJAX calls where the data being presented is “mostly static” (e.g. username, messages, etc.). The data can then be accessed in the client-side JavaScript and used as necessary.

Looking at the typical three-tier architecture, Unity is a bit of glue that sits above the app logic (technically, it could be thought of as part of the app logic, but it’s separated here for illustrative purposes).

app stack with glue

However, since the glue makes 1+ calls to fetch the XML from the app logic, things look a bit more like this on the backend:

backend app stack, multiple, with glue

As I mentioned, this is all very simple stuff. Here’s the code that does the magic:

class Unity
{
protected $js = null;
protected $htmlFile = null;

function __construct($_htmlFile)
{
$this->js = array();
$this->js[] = "var Unity = {}";

$this->htmlFile = $_htmlFile;
}

public function PushXml($_xmlFile, $_namespace)
{
$xml = simplexml_load_file($_xmlFile);

// see http://www.php.net/manual/en/function.json-encode.php#97008 for why json_encode is done like this
$this->js[] = "Unity.{$_namespace} = " . json_encode(new SimpleXMLElement($xml->asXML(), LIBXML_NOCDATA));
}

public function PushVar($name, $var)
{
$snippet = "Unity.{$name} = {$var}";
$this->js[] = $snippet;
}

public function UnifyAndEcho()
{
$scriptBlock = implode(';', $this->js);

$dom = new DOMDocument();
$dom->loadHTMLFile($this->htmlFile);
$head = $dom->getElementsByTagName("head")->item(0);

$scriptElem = $dom->createElement('script', $scriptBlock);
$scriptElem->setAttribute("type", "text/javascript");

if($head->firstChild)
{
$head->insertBefore($scriptElem, $head->firstChild);
}
else
{
$head->appendChild($scriptElem);
}

echo $dom->saveHTML();
}
}

Note that you don’t have to parse the XML; json_encode() takes the SimpleXMLElement object and converts the entire tree to JSON. This is done in order to:

  • keep Unity (and the glue layer) as brain-dead simple as possible
  • avoid mucking around with application logic at this level (if the XML doesn’t fits well [e.g. too big, too much unnecessary data, etc.] it’s a problem at a lower level)

Now an example of how the Unity class would be utilized:

$uni = new Unity("test-page.html");
$uni->PushXml("http://localhost/test-xml.xml", "Account");
$uni->UnifyAndEcho();

Assume the XML output is:

<?xml version="1.0" encoding="UTF-8"?>
<root>
<account_info>
<username>
pixel</username>
<stats
timespan="month">
<check_ins>
12432</check_ins>
<photos>
123</photos>
<followers>
32</followers>
</stats>
</account_info>
</root>

… in the client-side JavaScript code, we can access any of this data via Unity.<namespace>… The namespace was specified as “Account”, so the variables can be accessed as follows,

// username
Unity.Account.account_info.username;

// stats.check_ins
Unity.Account.account_info.stats.check_ins;

// ...

Sessions

A not-too-obvious challenge in doing something like this handling session data. As Unity is calling scripts from localhost (and not the client’s computer), a different session, one for localhost, is active. In addition, session data is locked, preventing two active scripts from accessing the same session data (which the xml generating script may very well need)

First, here’s how to get the session id:

session_start();
$sess_id = session_id();
session_write_close();

session_write_close() closes the session so that the subsequent script call will be able to access it. If necessary, session data can be started and access again after or in-between Unity::PushXml() calls; however, again, Unity is meant to be just a lightweight bit of glue, there shouldn’t really be a need to read or manipulate session data at this point in the code.

We can now pass the session id as a GET parameter:

$uni->PushXml($localbase . "controller/getlogin.php?sid={$sess_id}", "Account");

Now within the script being called, session_id() needs to be called with the passed session id, before session_start().

if(isset($_GET['sid']))
{
session_id(
$_GET['sid']);
}

session_start();

That’s it.

Finally it’s worth mentioning that none of this is bound to PHP nor XML. This could be done in any server-side language and the app logic could just as easily output JSON.

Baidu Maps

Truly impressive: Baidu (“the Google of China”) doesn’t use satellite imagery for its maps, instead it uses isometric, Simcity-style sprites.

Baidu map of Shanghai

It’s beautiful to look at, and with the crystal clear delineation of buildings, roadways, and landmarks, it’s likely a very effective navigational tool as well. The sheer scale of this blows my mind. I can’t even begin to fathom how much time and effort went into creating this much artwork to paint an entire city.

Bing’s bird eye view is the only service I can think of that comes close to this, but is far from impressive at times,

Bing map of Shanghai

h/t ronnestam.com, VisualJournalism

#NewTwitter API

So a few interesting, or rather frustrating things, about the current state of the Twitter API.

I spent way too much time fucking around with a couple of PHP Twitter API libraries to do some automated tweeting. The change late last year of requiring OAuth authentication seems to have left most libraries broken. TwitterOAuth seemed to support OAuth and I saw it mentioned in a couple of blog posts, so I gave it a try. It seemed to work well and I got the access tokens, but subsequent requests resulted in a “Incorrect signature” error. So I used the Twitter class by Tijs Verkoyen, which did the trick.

With the new web client supporting media attach to tweets, I figured surely there must be some way to do this from the API. There isn’t. Only content from one of Twitter’s media partners is attached. The term “media partners” should have probably been a red flag to developers regarding the shape of things to come. To add insult to injury, the client-side code for this is ridiculously easy to extend (creating a new twttr.mediaType).

Finally, the recent announcement with Twitter coming out against third-party clients. I can’t say I’m surprised at all, especially as Twitter moves further towards selling ads and actually making a profit, for which they’ll need tighter control. Why third-party clients were allowed in the first place is questionable beyond it being a bait-and-switch tactic to lure developers and users. Personally, I use the web client, so I don’t care that much as a user and my recent development efforts have just revolved around server-side tweeting, so I don’t care that much as a developer, but to those burned I’d say suck it up and learn from it. Twitter, Facebook, bit.ly, whoever… pretty much every company out, they doesn’t make an API for your sake, but for theirs. Invest too much time, energy, and/or money and your almost definitely going suffer disappointment and betrayal. If your project is important to you, be ready to pull the plug and build your own platform or ecosystem; use them, don’t let them use you.

progTools 1.7

Another update to progTools. Ignore the jump in version number, that’s a result of fighting with the Air ApplicationUpdater.

New features:

  • Auto-update
  • Encode/Decode URI
  • Escape string
  • Encode/Decode HTML special entities
  • Text character and word counter
Please upgrade your Flash Player This is the content that would be shown if the user does not have Flash Player 9.0.115 or higher installed.

progTools 1.7

Better Checkboxes

Another take on styling checkboxes with jQuery+CSS by Martin Angelov.

Better Check Boxes with jQuery and CSS by Martin Angelov

I’m not sure if they should really be called checkboxes at this point. I did something very similar a while ago, I called them toggle buttons.

Vandalism by @vinusha

A few notes left on an unattended whiteboard; my sister leaves her own thoughts…

whiteboard

whiteboard

whiteboard

Quake 3

I have a bunch of old video game magazines I’m sorting through. I love the old issues of PC Gamer and it’s awesome to see some of the old ads and articles. Here’s a gem from the September 1999 issue:

quake 3 ad from Sept. 1999 issue of PC Gamer

I didn’t play Quake 3 until well after release, as the case is with most games I play, but for Quake 3 in particular I couldn’t play it until I got a Voodoo 3 as it required 3d hardware acceleration. I didn’t play online much, but I had a blast playing thru the single-player tiers. That said, I was disappointed with the lack of CTF, the dialog from the bots seemed stupid (attempting and failing badly to mimic how players talked online), and the lack of variety in level design was disappointing (transitioning only between the tech arenas and the hellish, fog-filled ones)… perhaps not an issue in multiplayer, but in single-player, the maps took a front seat to your soulless opponents. A solid game and great deathmatch experience nonetheless, but I remember playing Unreal Tournament much more, both online and off.

dotspott notes + submenu design update

Two very minor updates to the dotspott web client:

  • Sticky note icons now appears when a note is attached to a spott. Clicking the icon will bring up a dialog with the note.
  • Spott submenus are now made to look more like buttons and spaced a bit further down from the tag list.

spott with sticky icon and new submenu

Flexible Binary Format

Another project I worked on a long time ago but never released. The goal of Flexible Binary Format was to create a file format and library for dealing with (tree) structured binary data (in the same vein of what XML does for textual data). This was a reaction both to the verbose and non-binary-friendly nature of XML and to the ugly chunk-based structuring done in many binary file formats.

The code for the C++ library is now up.

I’ll do a high-level, quick-and-dirty tutorial in this post to show how things work.


include FBF.h

#include "FBF.h"


Build a data tree

FlexibleBinaryFormat::DataTree        dt;
FlexibleBinaryFormat::TreeBuilder    dataTreeBuilder(&dt);


Attach a node to the data tree

dataTreeBuilder.AttachNode("root/vertex");


Make a vector of nodes and sub-nodes

dataTreeBuilder.AttachNode("root/vertex");
dataTreeBuilder.AttachNode(
"root/vertex");
dataTreeBuilder.AttachNode("root/vertex[0]/xyz");
dataTreeBuilder.AttachNode(
"root/vertex[1]/abc");

Note: We created 2 nodes on the same path (root/vertex). We access them individually using an index number within brackets.


Attach data to nodes

dataTreeBuilder.AttachDouble("root/vertex[0]/xyz", 0.12345);
dataTreeBuilder.AttachDouble(
"root/vertex[1]/abc", 54321.0);

Once data is attached to a node, you can’t attach a sub-node to it.
(i.e. data is stored on the leaves of the data tree)

The following methods of FlexibleBinaryFormat::TreeBuilder are available to attach data to a node:

  • AttachString()
  • AttachBool()
  • AttachByte()
  • AttachSByte()
  • AttachDouble()
  • AttachFloat()
  • AttachShort()
  • AttachUShort()
  • AttachInt()
  • AttachUInt()
  • AttachInt()
  • AttachUInt()
  • AttachLong()
  • AttachULong()


Write the data tree out to a file

FlexibleBinaryFormat::FlexibleIO    flexio(&dt);
flexio.WriteToFile(
"test.fbf");


Read a data tree from a file

FlexibleBinaryFormat::DataTree        dt2;
FlexibleBinaryFormat::FlexibleIO    flexio2(&dt2);
flexio2.ReadFromFile(
"test.fbf");


Read data from a node

double v0xyz = dt2.GetData<double>("root/vertex[0]/xyz", 0);
// second arg is always 0;
// arg is a leftover from some deprecated stuff in older versions and will be removed

progTools 1.3

An update to progTools is now available.

New features and fixes:

  • Corrected ISO-8601 timestamp format
  • Added MySQL timestamp format
  • Base64 encoding and decoding
  • File to data URI generator
Please upgrade your Flash Player This is the content that would be shown if the user does not have Flash Player 9.0.115 or higher installed.

progTools 1.3