2 August 2006
How to Measure the Success of your Web App
So you are building a web application… good for you! How are you doing? “Lots of people have signed up so we’re doing really well”, that sounds nice in a “you probably have no clue” kind of way. Allow me to explain.
Part of the benefit of doing business online is you can track EVERYTHING. Web services are the pinnacle of such businesses in my eyes because they are eminently trackable. But the questions you will be asking are: what to track, and where to start, and why even bother. In this article I’ll show you how to approach these questions, so you can find out (or at least have a much better idea) how your web application really is doing.
Why Track Anything?
The Eisenberg Brothers at Future Now Inc. have put together a great body of work preaching the message, not for eyeballs and traffic, but rather for conversions. To explain why conversions are so important to your business, the Eisenbergs introduced the concept of the leaky bucket. Very simply stated, if you have a leaky bucket, to keep it full you can keep adding water, or you can patch the hole. If the bucket represents your website, and water represents your sales, you can keep adding traffic to keep your sales numbers high, or you can patch up the holes in your website and increase the number of sales with your existing traffic. Constantly adding more traffic is not sustainable for many small businesses (especially start-ups with a $0 marketing budget), so the only option is to patch up the bucket.
The Conversion Funnel
A good way to start tracking all the different types of conversion for your business is to put them together in a funnel diagram like this:

This is a classic sales funnel for a web application. At each stage of the funnel, you can expect to lose a percentage of your visitors. So, let’s say 10 per cent of the people who visit your website trial your product, and let’s assume 10 per cent of those people decide to pay for your service. Therefore you are in fact losing 90 per cent of your visitors at each step of the process. This is actually not a bad conversion rate for many services and in this scenario your sales funnel would look like this:

To sum up, if you lose people at every stage – and again, it is a fairly common scenario for you to lose 90 per cent at each stage with a web service – then your bucket is leaking.
A More Sophisticated Web Service Funnel
If your bucket is leaking, then you need to identify the largest leaks and plug them fast. Breaking up your funnel into smaller pieces can help. Here is a more precise funnel, useful for services such as FreshBooks, Basecamp or DropSend:

Whoa… how did things get so complicated? They actually aren’t. Comb over that list and you will see these are merely the general steps a user must go through before they pay you for your web service. Once you see this funnel laid out on the page, identifying your weak spots becomes easier.
How to Track Your Progress
Let’s assume you commit to tracking and defining each of the six number of steps I laid out above. Here’s how we do it with FreshBooks:
| Metric | Description | How to Track (Tool) | Units |
|---|---|---|---|
| Visitors | First time unique visitors to your website | Remotely hosted web site analytics service | First time unique visitors |
| Trials | People who sign up to trial your service | Remotely hosted web site analytics service AND in-house database | Signups |
| Logins | People complete all your registration steps and actually login to their account | In-house database | Auto increment a “number of logins” field each time they login. “0″ for did not login. |
| Active Users | People who have used the service recently (ie. in the last 2 months) and have logged in a given number of times (ie. logged in at least 10 times) | In-house database | “Number of logins” greater than some number you choose (for example, “greater than 10 logins”). |
| Paying Users | People who pay for you your service | In-house database | Number of users/systems/accounts who pay you for your service. |
| Staying Users | People who continue to pay for your service for more that 12 consecutive months | In-house database | Number of users who have been paying you for more than 12 months. |
To track step #1 (visitors), go get yourself a good, remotely hosted JavaScript (not server log) analytics solution. Google Analytics is a free example. We recommend IndexTools. If you can afford enterprise software (or just like data and have the money) you might want to try Omniture. Again, I would avoid server log parsing solutions like AWStats, Webalizer and Urchin 5 because they are not as accurate as you need them to be.
You may have noticed in the chart above that we don’t use our stats to track anything but the first two steps in our conversion funnel: visitors and trials. That is true. I like the accuracy of database tracking – especially when your numbers are low (ie. you are just getting started) and inaccuracies can really throw you off. Tracking does create programming and database overhead, but since you are storing very little information it does not put much strain on your database, and you can always turn the tracking off at some point down the road. Also, and I would say this is much more important, analytics do not give you good active user counts. What does give you good active user counts is tracking the number of times a user logs in.
Stats are also AWFUL at tracking paying user upgrades for a web service, because the transaction is not immediate – it can take weeks and/or months – and sometimes one user will upgrade multiple times. For these reasons I highly advocate tracking these metrics yourself using your database. You will likely want to build some custom reports for yourself as well. Allocating developing resources for internal projects like this is part of a growing application development trend known as shadow application development. It’s an investment in yourself and it will pay off.
How to Decide What to Fix First
So let’s assume you take my advice and you build and track the conversion steps. And let’s assume your bucket is leaking (if it isn’t, email me with the subject “EAT MY SHORTS”). The question is, “what do I fix first?” The answer? Start as high up the funnel as possible.
Most people need to start with visitors to trials. If 5-10 per cent of first time visitors do not trial your service, you have some work to do – fast. If you need more specific guidance, I recommend adding another step to your conversion funnel. For example, if you can’t figure out where you are losing people before they sign up to trial your service, add another step in your stats called “view sign up”. If 30 per cent of your visitors are viewing your sign-up page, but only 2 per cent are actually signing up, you’ll want to redesign your sign-up page.
Here is a chart with some hints as to where to invest your time in an effort to improve each stage of your conversion funnel:
| Metric | Description |
|---|---|
| Visitors | Referral program, spend on marketing (Google AdWords), get links from friends etc. |
| Trials | Closely monitor your web stats. See what pages people commonly exit your site from and figure out why, then change that page. Track to see if people are making it to your “trial/sign up” page… if they are not, make sure you are clearly directing people to this page in appropriate places throughout your site. |
| Logins | Spend time making your forms friendly, and anywhere you can, reduce steps between your public-facing website and your account login. Reduce barriers. Handle form errors in a friendly manner. This step is easy to track, and done well, will turn your users into friends. |
| Active Users | Make your application simple to use and useful. Spend time watching people use your application. Do the “Mom Test” and get your Mom in there if you can. |
| Paying Users | Active users become paying users if you add enough value and get your pricing right. Pricing is tough – don’t be afraid to change your pricing down the road once you have more data about how your users work. |
| Staying Users | Deliver a great service with excellent support EVERY DAY. |
How Much Data Do You Need to Have a Decent Sample?
Ideally, you want 100 or more records so you have a decent sample in any stage of your conversion funnel. But think about that for a second, if you need 100 paying subscribers, and you convert one per cent of all site visitors to paying users, then you need to generate 1,000 trials, and drive 10,000 visitors to your website. That can take months when you are just getting started. This is why it is so important to start focusing on the top of that funnel first (ie. visitors to trials, the trials to successful registration form completion). These metrics are the fastest to track, and if you get them set right, you will get more people into your application who will then give you feedback so you can work on your other metrics. It’s a virtuous circle.
Conclusion
You can’t improve what you can’t measure, so start measuring your conversion funnel ASAP. It’s pretty remarkable when you see the results, and if you take the time to do it, you will know how you are doing in no uncertain terms.
Shout out to those of you who run web services: I’d love to know how your conversion funnel is doing so that I can aggregate some data and share it with entrepreneurs who are trying to get started. Shoot me an email if you would like to participate. Thanks.
Like this article? Digg it!

We're big fans of 


MiniSites
# April 26, 2009 - 7:31 pm
Great info Michael .. The diagram is spot on!
Vedat
# May 8, 2009 - 7:18 pm
thank you
sohbet
# June 10, 2009 - 11:14 am
thanks admin good post
Recenzje. Serwisy społecznościowe. Projektowanie « O programowaniu i nie tylko…
# August 20, 2009 - 10:49 pm
[...] rozdział jest poświęcony metodzie analizy lejka zmian — sposobowi mierzenia jakości poszczególnych opcji aplikacji (np. kroków rejestracji) lub [...]