Any sufficiently advanced technology is indistinguishable from magic.
- Arthur C. Clarke

When Google first unveiled GMail, then GMaps, a firestorm of interest and activity was generated — not just in those Google applications, but the technologies that powered them.

These web programming techniques (now, of course, known collectively as AJAX) have become incredibly popular, especially among Web 2.0 startups and their early adopters.

The developers of Ruby on Rails recognized early on that allowing RoR developers to easily AJAX-ify their webapps would be a great addition to the framework.

With the addition of RJS Templates to Rails core, the AJAX bar was lowered even further.

Simple AJAX requests like incrementing the number of diggs a story has received, or splicing a comment into a blog, are remarkably fast, not to mention user friendly.

What we’ll be addressing today, though, is optimizing ever more complex AJAX requests that might involve a multitude of SQL calls and JavaScript rendering techniques.

Not all of the techniques are strictly JavaScript/browser-oriented. The list was actually derived from the steps that we took to optimize Mailroom, the first app in the Sproutit suite for small businesses.

Mailroom is built for use with small teams; it can be used to manage shared email accounts such as support@ and sales@. It combines an ajax-rich front end with features like tagging, saved replies (for later reuse) and shared internal conversation notes.

When we first launched Mailroom, the service was reasonably fast. As we grew and our server got busier, however, some customers started to complain about how long it took for some of our AJAX functions to complete.

The following optimization techniques, though not complete by any means, were just a few of the methods we used over the course of several weeks to bring Mailroom, literally, up to speed.

Following the performance boost, Mailroom’s response times for a Load Conversation call (via AJAX), went from 2-8 seconds to averaging less than half a second. Once we added pre-caching (technique #5) on the client side via JavaScript, clicking on a message resulted in a near-instantaneous load; sometimes faster than Outlook or Apple Mail.

Possible Sources of Latency

When building out your AJAX application, it’s a good idea to keep in mind the different sources of latency that can slow down the user experience. These include:

1. The Database (both in memory and filesystem swap)
2. App Server Execution & Rendering
3. The Webserver
4. The Network
5. The Browser (only in really complex AJAX apps)

There are others, but we hope this broadly covers them all. We’ll only be taking a look at a few of these sources, but it’s useful to keep them all in mind as you build out your app.

5 Ways You Can Optimize Your Ruby on Rails/AJAX Application

1. Optimal Database Indexing
2. Eliminate Redundant SQL Queries
3. Fragment Caching
4. Response Text Compression/Minimization
5. Pre-rendering and Client-side JavaScript Caching

1. Optimal Database Indexing

For many applications, database indexing will be the biggest performance booster of them all.

Sample output from a Rails log file:

  Completed in 8.29523 (0 reqs/sec) | Rendering: 2.68176 (32%) | DB: 5.38202 (64%) | 200 OK [http://railsapp.org/action]

This action would be an ideal candidate for database optimization. If, however, you see something like DB: 0.05 (5%), the action is already spending most of it’s time rendering and there’s probably not a lot you can do in the way of database indexing.

It used to be somewhat difficult to pinpoint the exact function call in your Rails app that was generating an SQL query. But with last year’s release of Nathaniel Talbot’s QueryTrace plugin for Rails, this query back tracing process got a whole lot simpler.

After you install the QueryTrace plugin your Rails app (in the ‘development’ environment) will give you output like this in your log/development.log:

 Conversation Load (0.001538)   SELECT * FROM conversations WHERE (conversations.id = 23453) LIMIT 1
    app/models/feed_observer.rb:14:in `after_create'
    app/controllers/conversation_controller.rb:162:in `send_to'
    script/server:3

As you can see, it’s now trivial to find the exact call that’s generating the SQL database hit.

Adding Indexes via Migrations

The following is a migration’s self.up method to both create an articles table and add a few indices:

def self.up     # brings db schema to the next version
  create_table :articles do |t|
    t.column :title,      :string
    t.column :author_id,  :integer
    t.column :body,       :text
    t.column :is_live,    :boolean, :default => false
  end
  add_index :articles, :author_id
  add_index :articles, [ :author_id, :is_live ], :name => 'author_live_idx'
end

This would create both a single-column index on the author_id, as well as a multi-column index on author_id and is_live.

When in doubt, use a tool like railsbench or the Ruby Performance Validator to see where the bottlenecks are in your Rails app.

2. Eliminate Redundant SQL Queries

Note: this one is more of a “gotcha” applicable to those of us who are new to Ruby on Rails. (Since I’ve fallen victim to this mistake several times, I thought it was worth mentioning here.)

It’s always a good idea to keep a ‘tail -f log/development.log’ open when you are developing your Rails apps. (Win32 users can get tail, etc. here or use Cygwin )

You might be surprised at what you find if you are not in the habit of checking your query logs.

Ever written a quick hack like this?

  class ApplicationController < ActionController::Base
    def current_member
      return Member.find(@session[:member])
    end
    helper_method :current_member
  end

Only to discover that you’re actually calling current_member more than once in a particular controller/view combination? (thus generating multiple SQL calls for the same unchanged ActiveRecord object)

Here’s how to cache the variable in a per-request instance variable so that the Member.find(session[:member_id]) database query only gets called once per request:

  class ApplicationController < ActionController::Base
    protected
      def current_member

        @current_member ||= Member.find(session[:member_id])
      end

      helper_method :current_member
    end
  end

3. Response Text Compression/Minimization

Before attempting the below technique, the simplest way to reduce the number of bits sent across the wire is to enable gzip compression in your web server.

Some links on gzip compress: mod_deflate (apache 2.2), thread on config’ing lighty with gzip, apache 2.2 mod_deflate tutorial.

In the initial version of Mailroom, we were loading the tag conversation HTML module via AJAX for every load conversation request.

There was quite a bit of superfluous HTML that was being returned, when all we really needed was the specific tags that this particular conversation had been tagged with.

By the way, this kind of optimization is helpful when you have one master page load (where it’s OK to take 1-2 seconds longer), followed by many successive AJAX requests that you want to be rendered as quickly as possible.

We’re also targeting the size of the AJAX response text sent back by the server. A 4k response text is of course more preferable to a 16k one.

Initially, this was the HTML snippet being returned for every load conversation AJAX call:

Choose from the tags below:


         … lots of tags  …
    

or enter some new tags:


 …
    

Instead of returning this whole chunk of HTML each call, we are going to load a static, hidden HTML div once on the initial page load, and return (via AJAX, each call) just the bare essential tag data needed to render this static HTML into a per-conversation tag editor.

First we create a simple Tags object in javascript (with a little help from Prototype):

  Tags = Class.create();
  Tags.prototype =  {
    initialize: function(conversation_id, tag_str, tag_ids) {
      this.conversation_id  = conversation_id;
      this.tag_str          = tag_str;
      this.tag_ids          = tag_ids;
    }
  }

In your main page load, we render a static version of the tag editor template that stays in a hidden div.

The static HTML in a hidden div:

        Tags: #TAGS# 
      … a lot more static HTML.  this olds a static list of tags   checkboxes for this organization  …

When a conversation is loaded, the contents of the hidden div will be read in via JavaScript. Regular expressions are performed against it on the fly to replace elements like #TAGS# with actual per-conversation data returned via the AJAX request.

Here we create a simple JavaScript function that pulls in the contents of the hidden div, performs regex replacements on it, and completes with a Prototype Insertion call of the newly rendered HTML.

  loadTagEditor: function(tags) {
    var tags_static = $('tag-editor-static').innerHTML;
    var regex = /-static/g;
    tags_static = tags_static.replace(regex, '');  # Strip out the "-static" portion of the ids

    var tags_regex = /#TAGS#/;
    tags_static = tags_static.replace(tags_regex, tags.tag_str);
    ...
    new Insertion.After('the-widget-before-tag-editor', tags_static);
  }

Now whenever a conversation is selected, we just need to make sure the “loadTagEditor(tags)” method is called.

There are a few gotchas you’ll want to watch out for with this technique, such as not ending up with duplicate ids in your HTML.

In the hidden, static divs, one way to avoid duplicate ids is to append all your id names with ”-static” and then strip out the ”-static” when you load in your hidden div HTML. (that’s what we do above with the Regular Expression)

Now, our original chunk of HTML (which can really add up if you have 20 – 50 tags) that was previously being returned back with each Load Conversation AJAX request—is now replaced with a few simple lines of JavaScript:

  tags = new Tags(460857, 'home Marketing Development', new Array(1251, 1252, 1460));
  loadTagEditor(tags);

This is the Rails RJS (inside show_conversation.rjs, perhaps) that would be used to generate the above JavaScript:

  page << "tags = new Tags(#{@conversation.id}, '#{@conversation_tags * ' '}', new
Array(#{@conversation_tag_ids.inspect_raw}));"
  page << "ch.loadTagEditor(tags);"

4. Fragment Caching

Here we’ll use the ever so Web 2.0 example here of the tag cloud. These little beauties can be expensive to calculate in SQL. Once you’re site has been TechCrunched, you’ll want to make sure that your tag cloud (especially miniature versions loaded on heavily-trafficked pages, or via AJAX) gets cached as a Rails HTML fragment.

  class TagController < ApplicationController

    def tag_cloud
      fragment = read_fragment("myrailsapp.com/tagcloud")
      if not fragment
        setup_tag_cloud
        fragment = render :template => “tag/tag_cloud”
        write_fragment(”myrailsapp.com/tagcloud”, fragment)
      else
        #logger.info “Fragment cache read: #{fragment}”
        render :text => fragment and return
      end
    end

    private
      def setup_tag_cloud
        # Expensive SQL queries here …
      end
  end

Uncomment the logger.info call to verify that your fragment really is being read from the cache!

Admittedly, fragment caching in Rails can make your code less maintainable. It’s a technique that should really only be used for expensive-to-compute, fairly static portions of your site.

There are several helpful rails caching plugins on the Plugins page on the RoR wiki.

One simple but effective plugin is the Timed File Store which allows you to set an expiration (e.g. 15 minutes) on your cached fragments.

5. Client-side JavaScript Caching and Pre-rendering

If you really want to wow your users, pre-cache commonly called AJAX components into hidden divs so that the only time necessary to load them is the time it takes their browser to execute (eval) the pre-rendered JavaScript.

In the following example, we’ll cache conversations into hidden divs so that whenever a user clicks on a conversation, it’ll load almost instantaneously.

The pre-caching functions will all access a single global JavaScript variable that holds an array. That array will be populated on the first page load with the conversation IDs that should be cached.

Here’s the inline JavaScript (placed at the bottom of the main rendered HTML page):

Here are the JavaScript functions that do the preloading/caching:

  // Uses JavaScript's "setTimeout" function to call the "loadFirst" method (once per id).
  //    This method handles the logic of delaying the first load (to give the parent HTML page
  //    time to load) and putting a delay between each AJAX load so as to not overwhelm
  //    the browser & server.
  function preLoader() {
    var cnt = 0;
    var offset = 250;              # Milliseconds to delay the initial load
    var delay_per_load = 500;      # Milliseconds between each load
    ids.each(
      function (id) {
        setTimeout("loadFirst()", (offset   (cnt * delay_per_load)));
        if (id > 0) {
          cnt  = 1;
        }
      }
    );
  }
  // Pops off the first element in the 'load_ids' array and loads it
  function loadFirst() {
    var id = load_ids.pop();
    if (id > 0) {
      preloadConversation(id);
    }
  }
  // Preloads the conversation specified by creating a new AJAX request -- Note the use of Prototype's
  //   "onComplete" method to specify that "cacheConversation" should be called when the AJAX request
  //   has been completed and its data returned.
  preloadConversation = function(conversation_id) {
    new Ajax.Request('/mailroom/conversation/'   conversation_id   '?preload=true', {asynchronous:true, evalScripts:false, onComplete: cacheConversation});
  }
  cacheConversation = function(originalRequest) {
    // ... omitting some "clever" hacks used to get the conversation id ('conv_id' variable) ...
    var hidden_div_id = 'pre-'   conv_id;
    // Set the Inner HTML of a hidden div to the JavaScript passed back that was generated by Rails RJS templates:
    $(hidden_div_id).innerHTML = encode(originalRequest.responseText);
  }

The following is the JavaScript function used to either A) load the cached conversation, or B) if the cache is not present, load the conversation directly off the server:

  loadConversation: function(id) {
    if ($('pre-'   id).innerHTML != '') {
      var decoded = decode($('pre-'   id).innerHTML);
      eval(decoded);
    } else {
      // Load the conversation via a new Ajax call (eval'ing the result by setting "evalScripts:true",
      //    then caching it via the onComplete parameter).
      new Ajax.Request('/mailroom/conversation/'   id, {asynchronous:true, evalScripts:true, onComplete: cacheConversation});
    }
  }

Phew. Well, there you have it. JavaScript, long the domain of cut n’ paste HTML jockeys, has proven in recent years to be a formidable challenger to Flash (and Applets, remember those), once hailed as the future of interactive web applications. Of course, each has their place and own unique advantages and disadvantages.

While most applications wouldn’t need the kind of firepower described in this technique, hopefully you’ve learned a little bit about JavaScript, Rails and RJS along the way.

Footnote

The inspect_raw method used in technique #5 is a custom method we added to the Array class. Go here to learn more about this method and how we implemented it.

Ajax has been a hot topic for quite a while now, and there seems to be a new book with the eponymous Greek warrior in its title pretty much every week. Coming in at just 207 pages, Jeremy Keith’s new book, Bulletproof Ajax, is the latest to join the fray. With only 200 odd pages how does he plan to cover such a complex subject you may be asking? Well, here’s the secret – Ajax is really quite simple when you get down to it.

Jeremy’s previous book, DOM Scripting, stands as a fantastic introduction to JavaScript in general and DOM Scripting in particular and Bulletproof Ajax follows on from that title to some degree. Although you get a short introduction to the JavaScript language it’s just enough to understand the examples (I’d recommend you have at least a passing knowledge of JavaScript before reading this book). Also if you are coming to Ajax as a server side programmer then this book is probably not what you’re looking for, but worth reading at a later date. Given the client-server nature of Ajax there are server-side code examples (in PHP) but these are generally brief and serve only to support the examples rather than look at real world usage. Again, it would be useful to have at least a passing familiarity with a server side language and to be able to know if you already have a web server handy to experiment with.

The book features plenty of sample projects – for instance a simple address book is built up, showing off how to use XML, JSON and HTML as data sources and introducing the central XMLHTTPRequest object. There’s lots of focus, as you would expect, on making these examples bulletproof, in this case making sure they work even if JavaScript is unavailable using a method called Hijax.

To go from simple inline event handlers through to completely unobtrusive, object-based code and discussions of closures in JavaScript would be pretty good going for a book twice the size. It’s testament to the clear, no-nonsense and eminently readable writing style that this never bogs down the examples. Some people are bound to complain about the use of the proprietary innerHTML property and the minimal coverage of XML and JSON in the larger examples. This seems to be a facet of the scope of the book and a pragmatic approach to the problem rather than an unintentional oversight. If you’re looking for an A-Z of building an enterprise Ajax application then you’re probably looking for another book (but you should read this one first anyway!).

An entire chapter is dedicated to Accessibility and Ajax, a hugely important subject and one I’ve not seen mentioned anywhere else to date. Although the chapter lacks equivocal conclusions (mainly because no one seems to have formed any yet) it raises all the important issues for discussion and debate and provides a solid set of references for further reading. Throughout the book everything is anchored on the importance of user experience, rather than simply using technology for technology’s sake; this makes Bulletproof Ajax stand out from the crowd of more technology-focused tomes on the subject.

As a standards savvy developer if you want to get up to speed quickly with the hows and whys of modern Ajax but don’t have the time to wade through an awful lot of blog posts then Bulletproof Ajax is worthwhile reading. If you already know what you’re up to then it’s a perfect book to recommend to your unenlightened colleagues. A perfectly digestible read for one of those long train journeys!

Book Name: Bulletproof Ajax
Publisher: New Riders
Author: Jeremy Keith
URL: http://bulletproofajax.com
Price: $34.99 USD Buy Bulletproof Ajax at Amazon and save 34% off the cover price!
Rating out of 5: 3.5