2 July 2007
Any sufficiently advanced technology is indistinguishable from magic.
- Arthur C. Clarke
When Google first unveiled GMail, then GMaps, a firestorm of interest and activity was generated — not just in those Google applications, but the technologies that powered them.
These web programming techniques (now, of course, known collectively as AJAX) have become incredibly popular, especially among Web 2.0 startups and their early adopters.
The developers of Ruby on Rails recognized early on that allowing RoR developers to easily AJAX-ify their webapps would be a great addition to the framework.
With the addition of RJS Templates to Rails core, the AJAX bar was lowered even further.
Simple AJAX requests like incrementing the number of diggs a story has received, or splicing a comment into a blog, are remarkably fast, not to mention user friendly.
What we’ll be addressing today, though, is optimizing ever more complex AJAX requests that might involve a multitude of SQL calls and JavaScript rendering techniques.
Not all of the techniques are strictly JavaScript/browser-oriented. The list was actually derived from the steps that we took to optimize Mailroom, the first app in the Sproutit suite for small businesses.
Mailroom is built for use with small teams; it can be used to manage shared email accounts such as support@ and sales@. It combines an ajax-rich front end with features like tagging, saved replies (for later reuse) and shared internal conversation notes.
When we first launched Mailroom, the service was reasonably fast. As we grew and our server got busier, however, some customers started to complain about how long it took for some of our AJAX functions to complete.
The following optimization techniques, though not complete by any means, were just a few of the methods we used over the course of several weeks to bring Mailroom, literally, up to speed.
Following the performance boost, Mailroom’s response times for a Load Conversation call (via AJAX), went from 2-8 seconds to averaging less than half a second. Once we added pre-caching (technique #5) on the client side via JavaScript, clicking on a message resulted in a near-instantaneous load; sometimes faster than Outlook or Apple Mail.
Possible Sources of Latency
When building out your AJAX application, it’s a good idea to keep in mind the different sources of latency that can slow down the user experience. These include:
- The Database (both in memory and filesystem swap)
- App Server Execution & Rendering
- The Webserver
- The Network
- The Browser (only in really complex AJAX apps)
There are others, but we hope this broadly covers them all. We’ll only be taking a look at a few of these sources, but it’s useful to keep them all in mind as you build out your app.
5 Ways You Can Optimize Your Ruby on Rails/AJAX Application
- Optimal Database Indexing
- Eliminate Redundant SQL Queries
- Fragment Caching
- Response Text Compression/Minimization
- Pre-rendering and Client-side JavaScript Caching
1. Optimal Database Indexing
For many applications, database indexing will be the biggest performance booster of them all.
Sample output from a Rails log file:
Completed in 8.29523 (0 reqs/sec) | Rendering: 2.68176 (32%) | DB: 5.38202 (64%) | 200 OK [http://railsapp.org/action]
This action would be an ideal candidate for database optimization. If, however, you see something like DB: 0.05 (5%), the action is already spending most of it’s time rendering and there’s probably not a lot you can do in the way of database indexing.
It used to be somewhat difficult to pinpoint the exact function call in your Rails app that was generating an SQL query. But with last year’s release of Nathaniel Talbot’s QueryTrace plugin for Rails, this query back tracing process got a whole lot simpler.
After you install the QueryTrace plugin your Rails app (in the ‘development’ environment) will give you output like this in your log/development.log:
Conversation Load (0.001538) SELECT * FROM conversations WHERE (conversations.id = 23453) LIMIT 1
app/models/feed_observer.rb:14:in `after_create'
app/controllers/conversation_controller.rb:162:in `send_to'
script/server:3
As you can see, it’s now trivial to find the exact call that’s generating the SQL database hit.
Adding Indexes via Migrations
The following is a migration’s self.up method to both create an articles table and add a few indices:
def self.up # brings db schema to the next version
create_table :articles do |t|
t.column :title, :string
t.column :author_id, :integer
t.column :body, :text
t.column :is_live, :boolean, :default => false
end
add_index :articles, :author_id
add_index :articles, [ :author_id, :is_live ], :name => 'author_live_idx'
end
This would create both a single-column index on the author_id, as well as a multi-column index on author_id and is_live.
When in doubt, use a tool like railsbench or the Ruby Performance Validator to see where the bottlenecks are in your Rails app.
2. Eliminate Redundant SQL Queries
Note: this one is more of a “gotcha” applicable to those of us who are new to Ruby on Rails. (Since I’ve fallen victim to this mistake several times, I thought it was worth mentioning here.)
It’s always a good idea to keep a ‘tail -f log/development.log’ open when you are developing your Rails apps. (Win32 users can get tail, etc. here or use Cygwin )
You might be surprised at what you find if you are not in the habit of checking your query logs.
Ever written a quick hack like this?
class ApplicationController < ActionController::Base
def current_member
return Member.find(@session[:member])
end
helper_method :current_member
end
Only to discover that you’re actually calling current_member more than once in a particular controller/view combination? (thus generating multiple SQL calls for the same unchanged ActiveRecord object)
Here’s how to cache the variable in a per-request instance variable so that the Member.find(session[:member_id]) database query only gets called once per request:
class ApplicationController < ActionController::Base
protected
def current_member
@current_member ||= Member.find(session[:member_id])
end
helper_method :current_member
end
end
3. Response Text Compression/Minimization
Before attempting the below technique, the simplest way to reduce the number of bits sent across the wire is to enable gzip compression in your web server.
Some links on gzip compress: mod_deflate (apache 2.2), thread on config’ing lighty with gzip, apache 2.2 mod_deflate tutorial.
In the initial version of Mailroom, we were loading the tag conversation HTML module via AJAX for every load conversation request.
There was quite a bit of superfluous HTML that was being returned, when all we really needed was the specific tags that this particular conversation had been tagged with.
By the way, this kind of optimization is helpful when you have one master page load (where it’s OK to take 1-2 seconds longer), followed by many successive AJAX requests that you want to be rendered as quickly as possible.
We’re also targeting the size of the AJAX response text sent back by the server. A 4k response text is of course more preferable to a 16k one.
Initially, this was the HTML snippet being returned for every load conversation AJAX call:
Choose from the tags below:
Instead of returning this whole chunk of HTML each call, we are going to load a static, hidden HTML div once on the initial page load, and return (via AJAX, each call) just the bare essential tag data needed to render this static HTML into a per-conversation tag editor.
First we create a simple Tags object in javascript (with a little help from Prototype):
Tags = Class.create();
Tags.prototype = {
initialize: function(conversation_id, tag_str, tag_ids) {
this.conversation_id = conversation_id;
this.tag_str = tag_str;
this.tag_ids = tag_ids;
}
}
In your main page load, we render a static version of the tag editor template that stays in a hidden div.
The static HTML in a hidden div:
When a conversation is loaded, the contents of the hidden div will be read in via JavaScript. Regular expressions are performed against it on the fly to replace elements like #TAGS# with actual per-conversation data returned via the AJAX request.
Here we create a simple JavaScript function that pulls in the contents of the hidden div, performs regex replacements on it, and completes with a Prototype Insertion call of the newly rendered HTML.
loadTagEditor: function(tags) {
var tags_static = $('tag-editor-static').innerHTML;
var regex = /-static/g;
tags_static = tags_static.replace(regex, ''); # Strip out the "-static" portion of the ids
var tags_regex = /#TAGS#/;
tags_static = tags_static.replace(tags_regex, tags.tag_str);
...
new Insertion.After('the-widget-before-tag-editor', tags_static);
}
Now whenever a conversation is selected, we just need to make sure the “loadTagEditor(tags)” method is called.
There are a few gotchas you’ll want to watch out for with this technique, such as not ending up with duplicate ids in your HTML.
In the hidden, static divs, one way to avoid duplicate ids is to append all your id names with ”-static” and then strip out the ”-static” when you load in your hidden div HTML. (that’s what we do above with the Regular Expression)
Now, our original chunk of HTML (which can really add up if you have 20 – 50 tags) that was previously being returned back with each Load Conversation AJAX request—is now replaced with a few simple lines of JavaScript:
tags = new Tags(460857, 'home Marketing Development', new Array(1251, 1252, 1460));
loadTagEditor(tags);
This is the Rails RJS (inside show_conversation.rjs, perhaps) that would be used to generate the above JavaScript:
page << "tags = new Tags(#{@conversation.id}, '#{@conversation_tags * ' '}', new
Array(#{@conversation_tag_ids.inspect_raw}));"
page << "ch.loadTagEditor(tags);"
4. Fragment Caching
Here we’ll use the ever so Web 2.0 example here of the tag cloud. These little beauties can be expensive to calculate in SQL. Once you’re site has been TechCrunched, you’ll want to make sure that your tag cloud (especially miniature versions loaded on heavily-trafficked pages, or via AJAX) gets cached as a Rails HTML fragment.
class TagController < ApplicationController
def tag_cloud
fragment = read_fragment("myrailsapp.com/tagcloud")
if not fragment
setup_tag_cloud
fragment = render :template => “tag/tag_cloud”
write_fragment(”myrailsapp.com/tagcloud”, fragment)
else
#logger.info “Fragment cache read: #{fragment}”
render :text => fragment and return
end
end
private
def setup_tag_cloud
# Expensive SQL queries here …
end
end
Uncomment the logger.info call to verify that your fragment really is being read from the cache!
Admittedly, fragment caching in Rails can make your code less maintainable. It’s a technique that should really only be used for expensive-to-compute, fairly static portions of your site.
There are several helpful rails caching plugins on the Plugins page on the RoR wiki.
One simple but effective plugin is the Timed File Store which allows you to set an expiration (e.g. 15 minutes) on your cached fragments.
5. Client-side JavaScript Caching and Pre-rendering
If you really want to wow your users, pre-cache commonly called AJAX components into hidden divs so that the only time necessary to load them is the time it takes their browser to execute (eval) the pre-rendered JavaScript.
In the following example, we’ll cache conversations into hidden divs so that whenever a user clicks on a conversation, it’ll load almost instantaneously.
The pre-caching functions will all access a single global JavaScript variable that holds an array. That array will be populated on the first page load with the conversation IDs that should be cached.
Here’s the inline JavaScript (placed at the bottom of the main rendered HTML page):
Here are the JavaScript functions that do the preloading/caching:
// Uses JavaScript's "setTimeout" function to call the "loadFirst" method (once per id).
// This method handles the logic of delaying the first load (to give the parent HTML page
// time to load) and putting a delay between each AJAX load so as to not overwhelm
// the browser & server.
function preLoader() {
var cnt = 0;
var offset = 250; # Milliseconds to delay the initial load
var delay_per_load = 500; # Milliseconds between each load
ids.each(
function (id) {
setTimeout("loadFirst()", (offset (cnt * delay_per_load)));
if (id > 0) {
cnt = 1;
}
}
);
}
// Pops off the first element in the 'load_ids' array and loads it
function loadFirst() {
var id = load_ids.pop();
if (id > 0) {
preloadConversation(id);
}
}
// Preloads the conversation specified by creating a new AJAX request -- Note the use of Prototype's
// "onComplete" method to specify that "cacheConversation" should be called when the AJAX request
// has been completed and its data returned.
preloadConversation = function(conversation_id) {
new Ajax.Request('/mailroom/conversation/' conversation_id '?preload=true', {asynchronous:true, evalScripts:false, onComplete: cacheConversation});
}
cacheConversation = function(originalRequest) {
// ... omitting some "clever" hacks used to get the conversation id ('conv_id' variable) ...
var hidden_div_id = 'pre-' conv_id;
// Set the Inner HTML of a hidden div to the JavaScript passed back that was generated by Rails RJS templates:
$(hidden_div_id).innerHTML = encode(originalRequest.responseText);
}
The following is the JavaScript function used to either A) load the cached conversation, or B) if the cache is not present, load the conversation directly off the server:
loadConversation: function(id) {
if ($('pre-' id).innerHTML != '') {
var decoded = decode($('pre-' id).innerHTML);
eval(decoded);
} else {
// Load the conversation via a new Ajax call (eval'ing the result by setting "evalScripts:true",
// then caching it via the onComplete parameter).
new Ajax.Request('/mailroom/conversation/' id, {asynchronous:true, evalScripts:true, onComplete: cacheConversation});
}
}
Phew. Well, there you have it. JavaScript, long the domain of cut n’ paste HTML jockeys, has proven in recent years to be a formidable challenger to Flash (and Applets, remember those), once hailed as the future of interactive web applications. Of course, each has their place and own unique advantages and disadvantages.
While most applications wouldn’t need the kind of firepower described in this technique, hopefully you’ve learned a little bit about JavaScript, Rails and RJS along the way.
Footnote
The inspect_raw method used in technique #5 is a custom method we added to the Array class. Go here to learn more about this method and how we implemented it.
