Posts Tagged ‘xml’

Unity

I did a little experiment with meshing XML + HTML + PHP.

First, a little background info. Most service-side web technologies (PHP, Ruby on Rails, etc.) were designed around the concept of applying server-side markup to HTML,

<title><?php echo $title;?></title>

… which is interpreted and processed when the page is pulled. There are various frameworks and templating systems to make this easier, cleaner, more structured, etc. but the concept is the same. AJAX comes into play here and there, but it’s seen mostly as a way to sprinkle bits of dynamic data across a page.

This model was fine a few years ago, but for modern web services and apps I think there needs to be a push away from this model, primarily for 2 reasons:

  • Large portions, if not most, of the content of many web apps is dynamic, AJAX-queried data.
  • Support for mobile and desktop platforms connecting to web services is becoming increasingly important; platforms/apps for which HTML/CSS (or at least HTML/CSS designed for a browser) may not serve to provide the best UI.

AJAX structured code fits the bill.

A 100% AJAX architecture elegantly separates application logic (output as XML) from presentation (output as HTML/CSS). Unfortunately, 100% AJAX is not that easy for a complex application. JavaScript interpretation can be slow, killing frontend performance, and backend performance can suffer from the repeated flood of asynchronous requests. In addition, the latency from a multitude of AJAX requests+responses can quickly become noticeable (especially with a burdened backend and during an initial page load). When such issues become apparent, the solution always seems to be to go back to the messy world of hacking server-side tags into HTML.

Unity is an attempt to maintain a 100% AJAX application framework, but alleviate the issues mentioned above for web apps. It’s very, very simple; just a bit of glue that takes the XML from one or more AJAX calls (on localhost; the application server itself), turns it into JSON, and embeds it within the <head> element of the HTML document being served. This should not be done for every call, but for AJAX calls where the data being presented is “mostly static” (e.g. username, messages, etc.). The data can then be accessed in the client-side JavaScript and used as necessary.

Looking at the typical three-tier architecture, Unity is a bit of glue that sits above the app logic (technically, it could be thought of as part of the app logic, but it’s separated here for illustrative purposes).

app stack with glue

However, since the glue makes 1+ calls to fetch the XML from the app logic, things look a bit more like this on the backend:

backend app stack, multiple, with glue

As I mentioned, this is all very simple stuff. Here’s the code that does the magic:

class Unity
{
protected $js = null;
protected $htmlFile = null;

function __construct($_htmlFile)
{
$this->js = array();
$this->js[] = "var Unity = {}";

$this->htmlFile = $_htmlFile;
}

public function PushXml($_xmlFile, $_namespace)
{
$xml = simplexml_load_file($_xmlFile);

// see http://www.php.net/manual/en/function.json-encode.php#97008 for why json_encode is done like this
$this->js[] = "Unity.{$_namespace} = " . json_encode(new SimpleXMLElement($xml->asXML(), LIBXML_NOCDATA));
}

public function PushVar($name, $var)
{
$snippet = "Unity.{$name} = {$var}";
$this->js[] = $snippet;
}

public function UnifyAndEcho()
{
$scriptBlock = implode(';', $this->js);

$dom = new DOMDocument();
$dom->loadHTMLFile($this->htmlFile);
$head = $dom->getElementsByTagName("head")->item(0);

$scriptElem = $dom->createElement('script', $scriptBlock);
$scriptElem->setAttribute("type", "text/javascript");

if($head->firstChild)
{
$head->insertBefore($scriptElem, $head->firstChild);
}
else
{
$head->appendChild($scriptElem);
}

echo $dom->saveHTML();
}
}

Note that you don’t have to parse the XML; json_encode() takes the SimpleXMLElement object and converts the entire tree to JSON. This is done in order to:

  • keep Unity (and the glue layer) as brain-dead simple as possible
  • avoid mucking around with application logic at this level (if the XML doesn’t fits well [e.g. too big, too much unnecessary data, etc.] it’s a problem at a lower level)

Now an example of how the Unity class would be utilized:

$uni = new Unity("test-page.html");
$uni->PushXml("http://localhost/test-xml.xml", "Account");
$uni->UnifyAndEcho();

Assume the XML output is:

<?xml version="1.0" encoding="UTF-8"?>
<root>
<account_info>
<username>
pixel</username>
<stats
timespan="month">
<check_ins>
12432</check_ins>
<photos>
123</photos>
<followers>
32</followers>
</stats>
</account_info>
</root>

… in the client-side JavaScript code, we can access any of this data via Unity.<namespace>… The namespace was specified as “Account”, so the variables can be accessed as follows,

// username
Unity.Account.account_info.username;

// stats.check_ins
Unity.Account.account_info.stats.check_ins;

// ...

Sessions

A not-too-obvious challenge in doing something like this handling session data. As Unity is calling scripts from localhost (and not the client’s computer), a different session, one for localhost, is active. In addition, session data is locked, preventing two active scripts from accessing the same session data (which the xml generating script may very well need)

First, here’s how to get the session id:

session_start();
$sess_id = session_id();
session_write_close();

session_write_close() closes the session so that the subsequent script call will be able to access it. If necessary, session data can be started and access again after or in-between Unity::PushXml() calls; however, again, Unity is meant to be just a lightweight bit of glue, there shouldn’t really be a need to read or manipulate session data at this point in the code.

We can now pass the session id as a GET parameter:

$uni->PushXml($localbase . "controller/getlogin.php?sid={$sess_id}", "Account");

Now within the script being called, session_id() needs to be called with the passed session id, before session_start().

if(isset($_GET['sid']))
{
session_id(
$_GET['sid']);
}

session_start();

That’s it.

Finally it’s worth mentioning that none of this is bound to PHP nor XML. This could be done in any server-side language and the app logic could just as easily output JSON.

Reflex Feedback widget

I worked on a small AJAX widget for user feedback built atop jQuery UI: Reflex Feedback. It’s inspired by the widgets you see from services like Get Satisfaction and UserVoice, but much simpler and it’s a frontend-only widget, how you handle the feedback info on the backend is up to you.

Here’s what it looks like.

reflex feedback widget dialog

And here’s what the tag that opens the dialog looks like:

reflex feedback widget tag

To use it, download or clone the ReflexFeedback repo from bitbucket

Place the .js file wherever you’d like but the /reflex.content folder should a subdirectory in the same folder as the page loading the .js file. Load reflex.js as you would any other javascript file:

<script type="text/javascript" src="js/reflex.js"></script>

Call Reflex.init() to add the widget to the page. The first argument is the DOM element to attach the additional HTML/CSS code to. The seconds argument is the server-side script to call when the user clicks Send Feedback.

Reflex.init($('body'), 'controller/post_feedback.php');

That’s it for the frontend. You should see the tag show up in the right-hand corner and when clicked the dialog open.

For the backend, the AJAX call to send the feedback info will send a POST request with 2 fields: feedback_type, feedback_txt.

Reflex expects an XML reply from the server:

<reflex>
<result>ok</result>
</reflex>

ok indicates a successful result, any other reply is considered an error.

A successful result will close the dialog and show another with a thank you message.

reflex feedback thank you dialog

For an error, a message is shown below the Send Feedback button, informing the user that an error has occurred and to try again.

reflex feedback send fail

As for what to actually do with the feedback, that’s up to you, but what I’m doing is sending myself an email with the feedback info. I’ve posted my PHP script below; feel free to use it, modify it, etc. If you do use this code, be sure to fill in your mail server credentials and a from address; you’ll also need PEAR’s Mail package installed.

<?php

require_once "Mail.php";
require_once "Mail/mime.php";

header('Content-type: application/xml; charset=utf-8');
echo "<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n";

if(!isset($_POST['feedback_type']) || !isset($_POST['feedback_txt']))
{
echo "<reflex><result>error:missing-arguments</result></reflex>";
}
else
{
$from = "...";
$to = "...";
$subject = "Feedback from user...";

$feedback_type = $_POST['feedback_type'];
$feedback_txt = $_POST['feedback_txt'];

$bodyHtml = "<html><body>";
$bodyHtml .= "<p>Type: {$feedback_type}</p>";
$bodyHtml .= "<p>Feedback: {$feedback_txt}</p>";
$bodyHtml .= "</body></html>";
$body = $bodyHtml;

$host = "...";
$port = "...";
$username = "...";
$password = "...";

$headers = array('MIME-Version' => '1.0rn',
'Content-type' => 'text/html; charset=utf-8',
'From' => $from, 'To' => $to, 'Subject' => $subject);


$smtp = Mail::factory('smtp',
array ('host' => $host,
'port' => $port,
'auth' => true,
'username' => $username,
'password' => $password));

$mail = $smtp->send($to, $headers, $body);

if (PEAR::isError($mail))
{
$err_details = $mail->getMessage();
echo "<reflex><result>error:send-failure</result><details>{$err_details}</details></reflex>";
}
else
{
echo "<reflex><result>ok</result></reflex>";
}
}

?>

That’s all for now. I’ll work on more features and options for customization in the future. You can see the widget in action over at dotspott.com

SlashXML (C++ version)

The C++ code for SlashXML is now up, it’s in the /cpp folder within the repository. The C++ version supports most of the functionality the C# version does; maintaining multiple codebases is very difficult!

The C++ version also has 2 dependencies: TinyXML, an awesome, light weight XML parser, and AString, a not-so-lightweight UTF-8 string class. I know not everyone might be crazy about using AString, especially as it’s fairly bloated; I fucked around with things like custom numeric to string conversions with this class – unnecessary stuff, and it’s probably slower than using sprintf. However, it’s well encapsulated in it’s own namespace. I’ll try to have something more elegant in the future.


Load an XML document from a file

SlashXMLReader    xmlreader;
xmlreader.LoadXMLFile(
"test.xml");


Load an XML document from a string

SlashXMLReader    xmlreader;
std::vector<
unsigned char> xmlData;
// copy octets from string into xmlData here
xmlreader.LoadXMLData(xmlData);


Retrieve data from an element

SlashXMLString data = xmlreader.GetData("root/child");


Retrieve data from an indexed element

SlashXMLString data = xmlreader.GetData("root/child[2]");


Get the number of child nodes under an element

int numKids = xmlreader.GetNumChildren("root", "child");

The SlashXMLReader::LoadXMLData() method is kinda messed up and bizarre. I don’t even remember what was going thru my mind there.

SlashXML

I recently put up the SlashXML library on bitbucket. SlashXML provides a way to retrieve data from an XML document via path names to elements. It’s somewhat in the same vein as XPath, but much simpler. I wrote this way back when because I was annoyed at the more conventional method of having to extract data during the depth-first traversal of the document tree – this is a fine way to parse the document and pull in the data but not necessarily the most elegant way to query it. I haven’t written any documentation so, as I did with the bluetooth stuff, I’ll post a quick-and-dirty tutorial here.

The C# code files and .NET DLL are up now (/cs folder). I do have a version of this in C++ which uses TinyXML, but there’s a major bug that needs to be resolved before I upload it.


Load an XML document from a file

SlashXML.SlashXMLReader xmlReader = new SlashXML.SlashXMLReader();
xmlReader.LoadXMLFile(filepath);


Load an XML document from a string

SlashXML.SlashXMLReader xmlReader = new SlashXML.SlashXMLReader();
xmlReader.LoadXMLString(xmlString);


Retrieve data from an element

string data = xmlReader.GetDataAt("root/child");

If the element doesn’t exist, has no data, or contains other elements instead of data, the return value will be null.


Retrieve data from an indexed element

string data = xmlReader.GetDataAt("root/child[2]");

Indexing on data read in occurs when there are multiple elements with the same name.


Get the number of child nodes under an element

int numChildren = xmlReader.GetNumChildren("root", "child");


Note that underlying SlashXML.SlashXMLReader is SlashXML.StringTreeUtility.StringTree which actually holds the data read in and which can be queried directly for the data. Also, SlashXML reads everything into memory – do not use this for massive files (e.g. 1GB+ documents)! In fact, you should probably reconsider your use of XML if your files are that massive.

You’ll notice a major missing feature here – retrieving data in attributes! It’s not too difficult to do, but I haven’t had time to do it. It’ll be in a future version.

XML DOM object to string conversion

One of those simple things which has eluded me for far too long. I finally found the solution here:

var string = (new XMLSerializer()).serializeToString(xmlobject); alert(string);

While the above is good to know, I’ve also discovered I don’t need it. I was looking to convert an XML DOM object, from an AJAX call, to plain text, but the XMLHttpRequest.responseText property provides just that. For jQuery users like myself, the XMLHttpRequest can be access via a synchronous jQuery.ajax call as the return value, or asynchronously via. the jQuery.ajax.complete callback or the jQuery.ajax.success callback (as of jQuery 1.4 the XMLHttpRequest object is now the third parameter).