8 July 2008
Matt Week – What was it like?
As you’ve all been hearing, we spent last week building Matt (Multi Account Twitter Tweeter). We’re absolutely thrilled with the way it went. We finished on time, we got some great feedback, and we learned a lot. But everybody’s been asking: What was it like to build it? What went wrong? How did you guys find Django? Why does Elliott spend all day snacking?
Building Matt was great fun. Python is a great language and Django is really good web development framework. We considered using other options (principally Symfony) and a language we both know well (PHP) but in the end we decided that it would be all part of the fun if we made it tricky!
Remember, this is our experience with Django and Python, and as a result it’s based on the comparison between these and our previous environments.
Cool points:
Python is fun to write and you end up with beautiful code. As the libraries pre-exist on the server a Django project can afford to be small and lightweight. It’s also very fast! Once we realised that Apache’s KeepAlive had yet to be switched off, The Mattinator ran like lightning, even with all the traffic from the launch.
When it came to coding, we noticed some differences between PHP and Python. The one I found interesting was the way in which Python allows you to change variables without using explicit “getters” and “setters”. But Python, like Ruby, has a very clever way of getting around writing all the code required. Tech-minded readers, see Ryan Tomayko’s example. The specific differences between the inner workings of both packages is another blog post in itself, and a techy one at that.
Template Filters in Django are a great way of elegantly including pure Python code into the light-weight template language. Using the |, or ‘pipe character’, you can filter data through a python function described in a special template tags library. This is a great example of the DRY principle – write a template tag once, and use it throughout your project. Django is even kind enough to come with a set of template tags ready for you to use. (Remember to uncomment the line in your settings.py file to enable custom template tag libraries or you’ll be tearing your hair out wondering why they aren’t being included.) You can then pipe your function through several different template tags to get the desired effect. For example, Django escapes HTML posted in your tweets. We used:
 tweet.tweet|force_escape|at_reply|safe
to escape the tweet status immediately – removing the HTML from the message so we can safely show it. We then add the @reply tags, then filter it through “safe” so as to avoid Django’s auto-escaping removing our special @reply links. Pipes are used in the UNIX command line to pipe data from one application to another, so they make sense in this context too.
Not-as-cool points:
I’m a big fan of convention over configuration, and it seems to me that Django is quite flexible whereas other frameworks – such Symfony and Rails – are more strict. While there’s a huge learning curve for strict frameworks, it definitely comes in handy when you have lots of developers working on a large project. New developers can just jump straight in and know exactly how everything works. A pre-defined structure is also a way of keeping things from getting out-of-hand later on. As many developers have found, it’s very easy to end up with a great big messy structure – imposing strict rules on your developers is a great way of keeping this in check.
There also aren’t any environments available in Django. In Symfony, environments for development and production are available straight out of the box. This means that you can customize settings for production, testing and development, as well as any other implementations you’re planning – like ’staging’ or ‘windows_machine’ or ‘elliotts_server_under_the_stairs_next_to_the_cat_food’. So you can edit the settings for all three environments locally, and keep exactly the same source code for all three versions, to be used automatically. With Django you have to set your source control method of choice to ignore your configuration files. Django only has one, settings.py, so this wasn’t too tricky – although we’d have appreciated being able to set up a settings-dev.py and settings-prod.py, or the equivalent, just for consistency.
And try as we might, we couldn’t get Python to connect to MySQL on OSX. If this is easily possible, please let us know! But we didn’t have much time, so we ran with SQLite3 on our local machines, to be switched out for MySQL on the live server. That was another setting we had to change for the live environment – but the Django ORM, which keeps all the database stuff separate, did its job flawlessly, and there were no SQL changes to make.
Coming from a PHP background, there are some great IDEs available, which are the editing programs used to edit your source files (mad props to Zend Studio) which read through your code and can auto-complete what you’re writing very intelligently. PHPDoc code, which is automatically written by Propel in the Symfony model files, can be read by the API in order to discover and propagate object types. This makes developing much easier, especially in a new language, because as you’re writing you get a list of methods available to you for any object. Although Textmate is a fantastic editor, this was something that I missed. Perhaps it’s a mark against me as a developer for not memorizing all the functions available in Django and Python, but I like to be reminded that I’m on the right track without having to switch to the browser to debug what I’ve written. This may be simply because PHP is such a wide-spread language that more tools have been built for it.
Things we discovered:
Turn KeepAlives OFF. When we first put Matt live, we had Apache’s KeepAlive directive set to its default, ON. This meant that Apache would keep its connections to each client computer alive, which meant that Matt ran slower, and slower, and slower. Some hosts may set this automagically for you – but it pays to check your httpd.conf settings just in case.
Conclusions:
If I’ve underestimated Django or Python, then please let me know! It’s only because we spent only four days working with both. All frameworks have their good and bad points – it comes down to a personal preference on language and structure. I think that the framework war will be waged for some time to come. The best advice I got was to pick a popular one, and get good at it. Once you get really good, you can do what you need to do very quickly and efficiently, and maintain it effectively. It’s all too easy to say “Framework X is better than Framework Y because Z” (as I’m sure we’re about to see in the comments below), but I think it’s getting to the stage where they’re all pretty good at spitting out HTML.
I like Django, and Python. I really do. They’re nice, and easy to use, very fast, and I don’t have any real problems with either. They’re tackling the opposition, an entrenched PHP developer-base, with great results. But it comes down to the real-world advantages of using them, and the real-world tools currently available – an abundance of PHP developers, great PHP IDEs, and easy PHP hosting. It also does come down to personal preference, and the requirements of the job. So there’s no real way of telling you which one to use.
But this is all set to change at any point. There are many other frameworks all vying for their share – what do you think?
Enjoy this article?
If you liked this article, feel free to re-tweet it to let others know. Thanks, we appreciate it :)
We love .net Magazine
We're big fans of .net so they've hooked us up with a rad deal: Save 50 percent on your subscription.
Alcides Fonseca
# July 8, 2008 - 6:15 pm
Hi, I’m a django developer, and I use environments, although there is not a default practice for this. I have settings.py that depending on the os.uname()[1] (machine name) I import settings_prod, settings_dev or whatever settings you want.
As for MySQL in OSX, many people find that difficult, here’s how to compile: http://hivelogic.com/articles/2007/11/installing-mysql-on-mac-os-x and a better pref panel here: ftp://ftp.mysql.com/pub/mysql/download/gui-tools/MySQL.prefPane-leopardfix.zip
Dan
# July 8, 2008 - 6:27 pm
Hi Elliott.
We develop mainly in Rails, and have Mac based Studio. MySQL has never been a problem to install (with the package direct from the site mySQL.com, and onto our Leopard MacBooks / Pro’s) and the supplied tools allow us to work through a GUI and also connect to our db servers at Media Temple.
Happy to share some experience if you want to drop us a note.
Matt looks great, we also have been working on a Twitter app over the last weeks and you can read our experiences at: http://blog.ocvision.com
We need to try Django one day. The Pownce story is a great case study for the language.
Dan
Brian Luft
# July 8, 2008 - 6:47 pm
You brought up a good point about managing Django projects across environments. I see this often as a point of confusion for newcomers to the framework – especially with respect to laying out project and app structure. It doesn’t help that the tutorials prompt the user to lay out the directory structure in a way that most seasoned users don’t use. I appreciate the flexibility afforded but I can imagine that it can be frustrating for a beginner trying to figure out what the “best” practice is for organizing code.
The settings files are another place where the Django devs decided to allow for individuality. A common practice is to manage machine/host specific settings in a separate local_settings.py file which is imported from settings.
It is always great to get a sense of how the initial Django experience shapes up for developers – especially devs familiar with other frameworks and building a real-world application. Thanks for sharing your thoughts with the community.
Ben
# July 8, 2008 - 6:58 pm
MySQL on OS X should be very straightforward, just a case of grabbing the latest x86 package (currently that’s here), and installing the package.
Does mysqld not start, or throw any errors?
Elliott
# July 8, 2008 - 7:08 pm
Hi Ben,
Sorry – I meant to say that we had trouble making Python work with MySQL on OS X. MySQL runs perfectly :) Thanks for your comment!
Peter Herndon
# July 8, 2008 - 8:32 pm
Here’s a great post on setting up local settings files. As for getting MySQL to talk to Python, I downloaded the SVN trunk version of MySQLdb and compiled it myself — I believe the released version won’t work terribly well on OS X. I am running Leopard, BTW, so that may have an impact. Python development is *much* easier on Leopard than on Tiger — no need to install a lot of stuff, as Apple includes a lot more.
Finally, let me point to Wing IDE. I’ve not used it myself, but I’ve seen it demoed at PyCon, and it looks exceedingly fit for purpose.
Other than that, your points are really very valid. I love Django, but the configuration is… large and messy. Logical, though.
Lawrence Oluyede
# July 8, 2008 - 9:02 pm
It’s common practice to have separate settings for separate environments (and maybe N loca ones for each developer). We, in a project, had:
- settings_production.py with all the keys/values
- settings_development.py with:
“”"
from settings_production import *
# CUSTOMIZATION OF ALL THE KEYS/VALUES
“”"
- settings_staging.py with:
“”"
from settings_production import *
# CUSTOMIZATION OF ALL THE KEYS/VALUES
“”"
and each developer had its settings.py (outside source control) with:
“”"
from settings_development import *
# CUSTOMIZATION OF ALL THE KEYS/VALUES
“”"
Hope this helps
Milan Andric
# July 9, 2008 - 3:24 am
When you say that KeepAlives ON meant that Matt ran slower, and slower, and slower … Don’t you really mean that your apache processes where filling up the apache request table according to MaxClients and not picking up new requests? I don’t see how a KeepAlive config could make the application slower. Just wrong choice of words or am I over-simplifying?
Re: “easy PHP hosting”
Python has been around for quite a long time, before PHP, as well as mod_python. People just haven’t been asking for it as much until recently, with web frameworks, and most decent providers have stepped up. Finding easy Django hosting is not a problem these days?
Shawn Blower
# July 9, 2008 - 6:02 am
I recently installed Python, Django, and MySQL on OS X. I also had trouble at first. It had to do with python-mysql. This website did the trick.
Elliott
# July 9, 2008 - 9:05 am
@Milan: Yes, that’s exactly right. I simply meant that the application stopped working. As the threads died, the page would load – but it took forever to load, and eventually simply stopped working. It looked as though the site was running slowly, but in fact it was just maxing out Apache’s request table as you said.
@Shawn: thanks for the excellent link – the problem is that I’m (perhaps mistakenly?) using MAMP. I might update the article to say so. Coming from the XAMPP on windows, which I’ve found to be excellent, I’ve found that MAMP has a few very interesting quirks, and this may be one of them. Having said that, it’s still a bit of a trick getting MySQL-Python to work on OS X, wouldn’t you say?
Elliott
# July 9, 2008 - 9:17 am
and @Lawrence, I like your solution, that would work. But really, it’s a bit of a workaround for useful functionality that isn’t present. In my opinion it would be more a bit elegant if the framework took care of that for you. I may be nit-picking :)
also @Milan regarding Python hosting – you’re right, Python is available with most good providers. But PHP hosting is definitely more wide-spread and available, as is PHP support… and I dare say more developers are picking up PHP than python. I don’t think this is a particularly good thing, though. It seems as though this is changing – Google App engine uses Python, for example.
Lawrence Oluyede
# July 9, 2008 - 10:01 am
@Elliott: eh eh python philosophy (and hence Django’s) is to give more control to the developer being more explicit. That’s why there’s less magic than in Rails.
Matt Courtney
# July 10, 2008 - 1:49 am
Very interesting post but it didn’t answer the question of why Elliott spends all day snacking.
James Bennett
# July 10, 2008 - 8:59 am
I find myself agreeing with Lawrence on setting up “environments”; the key thing I’ve found in teaching people Django is the realization that you need to learn to work with not just Django but Django and Python, because Django deliberately doesn’t reinvent a lot of things Python can already provide (better versions of) for free. So the natural thing for “environments” is just different settings files that you point to according to what you’re doing; since they’re just Python, they’re free to import from each other selectively and override things as needed.
Dave Marshall
# July 10, 2008 - 11:53 am
With regards to getters and setters, you can always use PHP5’s overloading magic methods (http://uk.php.net/manual/en/language.oop5.overloading.php)
I don’t think it’s as elegant, but has the same effect?
public $email;
becomes
private $email;
public function __set($prop, $val) {
$method = ’set’ . ucfirst($prop);
return $this->$method($val);
}
public function __get($prop) {
$method = ‘get’ . ucfirst($prop);
return $this->$method;
}
/**
* standard getters and setters
*/
Keir
# July 12, 2008 - 6:11 pm
@james Nice to see your comment on the blog – thanks for stopping by. Funnily enough I have been reading your book today and am finding it really useful.
I know the environments situation was something we didn’t quite get right this time but thanks to the comments here and some recent reading I think we will definitely try and implement something soon.
I think your point about working with Python as well as Django is really important here. Personally the lack of “magic” is one of the main attractions to the framework for me. I love the flexibility but can fully understand why some developers favour “convention over configuration”.
Keir
# July 12, 2008 - 6:19 pm
Apologies for those who posted and comments didn’t appear promptly. For some reason WordPress required them to be moderated, suspect the links got them flagged.
Greg Fuller
# July 13, 2008 - 10:22 am
I’m using MAMP with Django. The trick is to point DATABASE_HOST to the socket:
DATABASE_HOST = ‘/Applications/MAMP/tmp/mysql/mysql.sock’
Stephen Walli
# April 24, 2009 - 8:38 pm
I had a similar problem when I started experimenting with Django on the Mac. Here’s how I solved it:
http://stephesblog.blogs.com/my_weblog/2008/10/mac-django-mysq.html
Rahul Malik
# May 21, 2009 - 1:34 pm
I’ve been using Django on and off for personal and professional projects for roughly a year now. I would say that I stayed with it because I was much more comfortable with python as a language and I didn’t seem to hit any serious roadblocks with the framework itself with what i was trying to accomplish.
The problem, when the app gets large, is that I find the design of the application (not user interface but code organization) becomes more difficult and doesn’t seem to make as much sense. If you are developing a web application it seems that rails might be better suited with their strict conventions rather than trying to find a way to cleanly set up a set of reusable apps like django encourages.
for the settings environments i would just create separate settings.py files or check which machine or user is starting the application in order to determine which environment to load.
I’ll probably get back into rails (i started around the initial release) sometime soon and my main concern is developing a rails app that will be difficult to upgrade to future versions of rails. The django community cares deeply about backwards compatibility and their documentation is by far some of the best in the open source community.
feel free to email me for any django questions or concerns!
@rmalik