Blog :: Javascript Development and Google Modules

So I made my first Google Module (if this doesn't ring a bell, go here and note all the little boxes with headlines, the weather, etc. You can close em, drag them around, and if you have a google account you can add just the ones you want and have a busy homepage...) My module is really simple: basically it screen scrapes my sitemeter page and displays my hit total for the day and the last hour, and shows a list of my recent referrals. Very boring really; it looks like this:

You can actually add this as a module to your own Google homepage if you're interested by searching in the Module Directory for sitemeter (it hasn't shown up yet, but I submitted it, so I assume it will) or by clicking this Google Module  button: Once you've added it, it will refuse to display till you give it your sitemeter sitename (this is your username at sitemeter; mine is sm2metapundit, for example).

While fellow bloggers might enjoy having their stats displayed on their Google Homepage, developers might be more interested in the process I went through writing this thing. Google gives you a proxy api that lets you retrieve remote documents via javascript cross domain. I started out assuming that I would just whip together some regexes to extract the content I wanted. This actually worked pretty well if all I wanted was to show exactly the same structured html that Sitemeter displays. Unfortunately, their html is a little 2002ish: lots of nested tables and font tags, and Google Modules need to work in a little tiny bit of screen space.

I decided I should try to extract the actual content and at this point my 20 lines of regexes started to break down. It occurred to me at that regexes might not be the best tool for the job: maybe using the DOM would be easier.

Well, it wasn't. It was actually kind of painful: writing code that digs through a large tree (lots of document.getElementsByTagName('TABLE') type stuff) and repeatedly reloading the browser to see debugging statements print out what's going on... It was taking forever and I had just about decided that I needed to read the Venkman docs and break out a debugger when a mention of a "javascript shell" floated across my Bloglines.

It turns out that this is a truly awesome Javascript development tool. Via a bookmarklet you get a popup with a javascript "prompt". Unlike the Javascript Console included in firefox/mozilla, the shell has tab completion of methods, up/down history recall (like Bash or the Python shell) and allows you to make DOM calls on the document that was open when you clicked on the bookmarklet. This allows for rapid prototyping of javascript DOM I could interactively make document.getElementsByTagName calls and introspect the lists I got back. The total module I developed is only 75 lines or so (and of course I'm in trouble if Sitemeter updates their site) but it really sped up the process to be able to interactively step through the DOM of my target page. Very good stuff: if you do any JS development you should check it out.

Update: Hmm. Having used my module for a day I decided it would be a good idea to eliminate XSS attacks if possible. Yes, browser referrers are user input, and what do we say about user input? NEVER  trust it... Anyhoo, it's fixed.

Update: Welcome Sitemeter Blog readers. I made a couple more updates per David's suggestion: your Sitemeter Codename displays in the title now (in case you are monitoring several sites at once) and I tweaked the text in the preferences section . I'm also considering letting you add multiple Codenames and displaying just the summary line for each. Anybody interested? I welcome suggestions from people who are using this.

Oh yes, and I thought I should point out that you can only use the module if your site stats are public (Privacy Level "normal" in your sitemeter manager).

Update: Switched hosting of Sitemeter module away from Google Base due to changes in how Google displays html. The Install button above reflects the new change...

Posted on April 25th 2006, 12:51 AM