22 April 2008
I’m a front-end engineer slash designer who likes to think he can dabble in code. With my simple side projects I do a passable job and my hacks limps along, most likely inefficiently and always on the verge of breaking, but getting the job (mostly) done. But where I almost always trip up and get lost — in days of hair pulling frustration — is when I’m setting up my local development or live production environment. It’s so difficult to find an accurate, knowledgeable and up-to-date walkthrough for frameworks like Rails or Django that it’s almost comical. And if, by the grace of the server gods, I do finally get something that runs and deploys my code — well now I get to monitor it, keep it updated and running 24/7. Yay!
This is why Heroku immediately appealed to me when it went live. “Develop and deploy a Rails app entirely in the browser.” Yes please. Unfortunately, by the time Heroku launched I’d already been bitten by Python and Django and had moved on to build my side projects without Ruby or Rails. So back to the drawing board, setting up my environments, dying a little each day I lost — once again — rebuilding the wheel.
Then Google App Engine launched, and I’ve been gleefully experimenting ever since.
This two-part series of articles is for all those who, like myself, are comfortable with markup, design and app code, but would rather not bother with all the other stuff. You know, the stuff that keeps Evan Williams awake at night . I want to spend what little time I have for my side projects being a mostly clueless developer, not a completely clueless sysadmin.
This intro will get you up and running on GAE in literally minutes and walk you through a simple app that’ll show you most of everything you need to know upfront about what GAE can do, for you and your app.
The app we’ll be building with this article is a silly, web-industry-navel-gazing barely-an-app I created called Startupdown , “where startups throw down”. Two startups enter, one startup leaves. It was partly inspired by TechCrunch’s GAE proof of concept where you voted on random startups. But I like to think mine’s a little more fun, and just barely better designed.
Now, in order to get any GAE application running, you’re going to need to have an account. Unfortunately, right now there’s a waiting list of 10,000 or so to get into the beta. So we’ll have to just pretend we’re all in. And since we’re all in, we first need to create our application and name it. Mine’s named “startupdown”.
Next you’ll need to download and install the appropriate GAE SDK for your OS .
Then create the folder that’s going to house our project, I called mine startupdown. For this simple project our folder and file heirarchy is going to look like this:
- startupdown
- app.yaml – app’s basic settings
- index.html – app’s homepage
- index.yaml – auto-generated by GAE
- main.py – app’s main functions
- myloader.py – for populating our data, explained fully in Part 2
- results.html – takes the results from the from submission on the homepage
- startups.csv – used by myloader.py
- stlylesheets
- main.css
That’s all there is to it. So the first file we need to put into our app folder is the project’s app.yaml file — it holds the basic settings for our project — and it looks like this:
application: startupdown version: 1 runtime: python api_version: 1 handlers: - url: /.* script: main.py
Grab the completed app.yaml file here
Here “application” is whatever you named your app in the GAE admin. And we don’t need to worry about changing version, runtime or api_version, they can all stay as is. “Handlers” are GAE’s way of handling URLs, so right now our only handler is saying “whenever anyone pulls up the root URL or any subpages of it, we’ll use the main.py script to handle.”
Now that we’ve created our app.yaml file, we can fire up the local development server that comes with the GAE SDK. We start it like this:
dev_appserver.py ~/full/path/to/startupdown
This takes the contents of the project folder and gets it up and running at http://localhost:8080. And as we add functions and make changes to our app, the server will automatically load them, so we don’t need to worry about restarting it for every change.
With our environment now running we can get started on the meat of our application, the main.py file. The main.py is going to contain:
- all of our libraries
- all of our datastore models (I’ll explain what this is in just a second)
- all of our main page functions
First, we’ll start with our libraries, they look like this:
import wsgiref.handlers import os import random from google.appengine.ext import webapp from google.appengine.ext import db from google.appengine.ext.webapp import template
GAE apps need to import wsgiref.handlers and os in order to handle URLs, so just include those two lines by default. My app imports the random Python library because I’m using random numbers, as you’ll see in a few paragraphs. And the last three, webapp, db and template are for the GAE webapp framework, the GAE datastore API and the GAE template language respectively. You’ll most likely be including them by default in most of your GAE apps as well.
The next part of our main.py file is for our datastore models. One of the biggest reasons to use GAE is to take advantage of Datastore, its highly scalable data modeling API. Instead of using something like MySQL or PostgreSQL to create tables and fields in a relational database, you use the Datastore API to create data models like this:
... class Startup(db.Model): name = db.StringProperty() url = db.StringProperty() logo = db.StringProperty() class Vote(db.Model): loser = db.IntegerProperty() winner = db.IntegerProperty() ...
Here I’m defining the two data models I’m using in my Startupdown application. Every startup has a name, url and logo, which are all strings. And every vote has a loser and winner, which are both integers. For more complex apps you’ll probably have several models and many different types of properties, like floats, datetimes, blobs or any of the other types of data properties GAE supports .
Now that we’ve defined the type of data our application needs, we can write the functions that are going to be accessing and manipulating that stored data. Since this app is very basic, it only has two functions — one that displays the main page and one that takes the form values and displays the results of the voting.
Here’s the MainPage function:
...
class MainPage(webapp.RequestHandler):
def get(self):
startups = Startup.all()
int1 = random.randint(0,49)
int2 = random.randint(0,49)
while int2 == int1:
int2 = random.randint(0,49)
template_values = {
’startup1': startups[int1],
’startup2': startups[int2],
}
path = os.path.join(os.path.dirname(__file__), 'index.html')
self.response.out.write(template.render(path, template_values))
...
All this function is doing is 1) getting all the startups by calling Startup.all() (where Startup is the name of the Startup model we created earlier and all() is a function all models have that grabs all their data) 2) creating 2 unique random numbers between 0 and 49 (since we’re using these random numbers to pull 2 startups out of a list of 50 using the startup array, and don’t forget, arrays always start at 0) and 3) assigning the 2 random startups we grab to the template_values list (lists are a Python datatype, very similar to arrays, but you can put almost anything in them). Template_values are how you pass variables from functions to static HTML pages. The template_values array now has our two random startups and is available in our index.html file, thanks to those last two lines of code.
Speaking of index.html, let’s take a quick look at what our app’s front page looks like behind the scenes:
<html>
<title>Startupdown - Where startups throw down</title>
<head>
<link type="text/css" rel="stylesheet" href="/stylesheets/main.css" />
</head>
...
<table cellspacing="0" class="startups-table">
<tr>
<td><h2>{{ startup1.name }}</h2></td>
<td class="vs"></td>
<td><h2>{{ startup2.name }}</h2></td>
</tr>
<tr>
<td>
<a href="http://{{ startup1.url }}">
<img src="http://thinkvitamin.com/images/articles/startupdown/{{ startup1.logo }}" />
</a>
</td>
<td class="vs">VS.</td>
<td>
<a href="http://{{ startup1.url }}">
<img src="http://thinkvitamin.com/images/articles/startupdown/{{ startup2.logo }}" />
</a>
</td>
</tr>
<tr>
<td>
<form action="/results" method="post">
<input type="hidden" name="winner" value="{{ startup1.key.id }}" />
<input type="hidden" name="loser" value="{{ startup2.key.id }}" />
<p><input type="submit" value="Vote for {{ startup1.name }}"></p>
</form>
</td>
<td class="vs"></td>
<td>
<form action="/results" method="post">
<input type="hidden" name="winner" value="{{ startup2.key.id }}" />
<input type="hidden" name="loser" value="{{ startup1.key.id }}" />
<p></p><input type="submit" value="Vote for {{ startup2.name }}"></p>
</form>
...
Grab the completed index.html file here
Notice, first, how we reference variables inside HTML, using double curly braces around our objects and variables ( i.e. {{ startup1.name }}). Next notice how I refer to startup1 and startup2’s properties using dot notation. Remember when we defined the Startup model earlier as having a name, logo and url? Well now we’re accessing those properties using startup1.name, startup1.logo and startup1.url.
Also notice how, in our form, we’re using startup1.key.id — this gives us access to the ID, or key, that GAE automatically creates whenever we import or create data in the datastore. And, lastly, we set the form’s action to “/results”, which is the app’s only other page, and is also the only other function in our main.py file. Let’s take a look:
...
class Results(webapp.RequestHandler):
def post(self):
vote = Vote()
vote.loser = int(self.request.get('loser'))
vote.winner = int(self.request.get('winner'))
vote.put()
...
The first part of our Results function is what processes the form data. def post(self) is grabbing the post data and vote = Vote() is creating a new vote model so that we can add the vote data to the datastore. We then grab our form’s loser value with self.request.get('loser') and assign that to vote.loser. And we then do the same to vote.winner. Then, with vote.loser and vote.winner assigned values, we put that data into the datastore using the put() function.
Grab the completed results.html file here
So now we’ve grabbed the ID’s of the losing startup and the winning startup, next we need to grab the rest of the startup’s data.
...
theWinner = Startup.get_by_id(vote.winner)
theLoser = Startup.get_by_id(vote.loser)
q = Vote.gql("WHERE winner = :1 AND loser = :2", vote.winner, vote.loser)
winnerVsLoser = float(q.count())
q = Vote.gql("WHERE loser = :1 AND winner = :2", vote.winner, vote.loser)
loserVsWinner = float(q.count())
...
Grab the completed main.py file here
Using GAE’s get_by_id function, we can grab the rest of our startup’s info (name, url and logo) by using the ID’s we grabbed from the form. Notice also how we can use GAE’s query language, GQL, to get results from our Vote data. In the instance of the winnerVsLoser variable, we’re grabbing all the instances of Vote where winner equals the winner we grabbed from the form and loser equals the loser we grabbed. Then, using GQL’s count() function, we’re counting how many results were returned. If 5 people have voted for startup1 over startup2, we’d get 5 rows of data back from or GQL query.
Using queries like this we’re able to create percentages of how often certain startups won and lost against other startups, etc.
That’s a lot to take in for one article, so we’ll stop here. Next article, in the second part of Developing with GAE, we’ll look at how to import our list of 50 startups into Startupdown, how to deploy and administer our app once we’re finished and a few tips and tricks once you’re up and running.
