Archive for May, 2011

Blood & Chrome

hmm, my external hard drive enclosure looks a bit like a cylon,

external hard drive enclosure

CSV file detected as SYLK file in Excel

Working on some CSV export code and encountered this weird error when opening the CSV file in Excel,

Excel has detected that 'test.csv' is a SYLK file, but cannot load it. Either the file has errors or it is not a SYLK file format. Click OK to try to open the file in a different format.

The file was certainly a CSV file; I had no clue what an SYLK file was.

The issue, as I discovered here, is due to the first 2 letters in the file being “ID” (which was the name of the first column); change this to something else (e.g. “Id”) and the file loads correctly without any warnings or errors.

E: tzdata: subprocess installed post-installation script returned error exit status 1

I got the above error while attempting to install updates on a Linux Mint system. This thread provided a few possible solutions, none of which worked for me (though I was glad I discovered the grub » recovery mode » repair broken packages option, which might come in handy in the future). Digging a bit, I ran:

dpkg --configure tzdata

and noticed,

debconf: DbDriver "config": could not open /var/cache/debconf/config.dat

There was no /var/cache/debconf/ directory and, as indicated here, simply creating it would provide a fix. So a simple:

sudo mkdir /var/cache/debconf

… and I was good to go!

New aautar.digital-radiation.com index page

Finally did a new design for the aautar.digital-radiation.com index page:

aautar.digital-radiation.com index page redesign

I always hated the old design, but stuck with it for the longest while.

aautar.digital-radiation.com old index page

Net. Flix.

Netflix CEO Reed Hastings on streaming content:

“We had set up the whole business essentially for streaming, but the network wasn’t big enough years ago,” he said. “But in 2005 we clicked on YouTube and watched cats on skateboards — and we thought, it’s here! Since then, we’ve had so much fun finally delivering on our name: Net. Flix.

IMAP Pickup

An interesting little project I wanted to work on; I wanted to be able to pull attachments from emails in an IMAP mailbox and then download them. I wanted an IMAP solution instead of writing a script for the MTA as a script would be specific to the MTA software and not transferable to another server. In addition, there’s also the common case where you may simply not have access to the MTA.

The biggest help in putting this together and dealing with attachments was this blog post and this comment on the PHP docs. Information on doing this is a bit scattered and incomplete in many cases, likely because extracting attachments is somewhat difficult as email is a notoriously bad way to transfer files; the file data is base64 encoded and dumped in as part of the message body.

ImapPickup is the class that encapsulates all the necessary functionality,

class ImapPickup
{
protected $imapStream = null;

protected function findAttachments($part)
{
$partNum = -1;
$attachments = array();

$this->findAttachmentsRec($part, &$attachments, &$partNum, -1);

return $attachments;
}

protected function findAttachmentsRec($part, &$attachments, &$partNum, $partNumSub)
{
if (isset($part->parts))
{
foreach ($part->parts as $partOfPart)
{
$this->findAttachmentsRec($partOfPart, &$attachments, &$partNum, $partNumSub+1);
}
}
else
{

if (isset($part->disposition)){
if ($part->disposition == 'attachment') {
$attachments[] = array($part->dparameters[0]->value, $partNum, $partNumSub);
}
}
}

$partNum++;
}

public function getAttachmentContent($msgNum, $partNum)
{
$contents = imap_fetchbody($this->imapStream, $msgNum, $partNum, FT_UID);
return imap_base64($contents);
}

public function getAttachments($msgNum)
{
$struct = imap_fetchstructure($this->imapStream,$msgNum,FT_UID);
$attachments = $this->findAttachments($struct);

return $attachments;
}

public function getAttachmentsFromMessages($msgArray)
{
$msgIdToAttachmentsMap = array();

if ($msgArray)
{
foreach($msgArray as $msgId)
{
$attachments = $this->getAttachments($msgId);
if(!empty($attachments))
{
$msgIdToAttachmentsMap[$msgId] = $attachments;
}
}
}

return $msgIdToAttachmentsMap;
}

public function getMessages($searchQuery)
{
return imap_search($this->imapStream, $searchQuery, SE_UID);
}

public function connect($mailbox, $user, $password)
{
$this->imapStream = imap_open($mailbox, $user, $password);
}

public function disconnect()
{
imap_close(
$this->imapStream);
}

}

Here’s a little example of how it can be used. This will query all messages with “pickup::” in the subject line and print out the messageID of all messages with attachments, followed by the filenames of all attachments for that message.

$imapPickup = new ImapPickup();
$imapPickup->connect("{mail.hotspotdot.net:143}INBOX", "test@test.net", "pass123");

$messages = $imapPickup->getMessages("SUBJECT pickup::");
$attachments = $imapPickup->getAttachmentsFromMessages($messages);

foreach($attachments as $msgId => $attArr)
{
echo "<p>{$msgId} => ";

foreach($attArr as $attachment)
{
echo $attachment[0];
echo ",";
}

echo "</p>";
}

$imapPickup->disconnect();

The array for a single file attachment contains 3 entries:

  • [0] => filename
  • [1] => major part number
  • [2] => minor part number

getAttachments(), findAttachments(), and findAttachmentsRec() will return an array of such entries (or an empty array is there are no attachments). getAttachmentsFromMessages() will return a map from messageID => array of single attachments.

The part number (both major and minor) is needed to retrieve the contents of an attachment. For getAttachmentContent(), simply use the major number if the minor number is <= 0, or concatenate them with a period separating them (e.g. "2.3").