13 December 2006
Plan to grow
Sheesh kebab! We are reaping a whirlwind of pain for not having a proper dev environment setup for FOWA.
Here’s the problem: when we launched the site, it was just Gill (my wife and co-Director of Carson Systems) and I FTP’ing files up to the site and using PayPal to allow people to buy tickets.
The new site has a mini web app (“The Lounge” – which allows attendees to network), a proper e-commerce system and a lot more pages than the original. It was designed by Apples to Oranges and built by Dave Stone. It’s hosted on a Rackspace server with Plesk installed.
Here’s the problem – we didn’t setup a proper Subversion repository and deployment script. Updates are a total nightmare now as there are three different parties working on the site.
Plan ahead
Here’s what we should have done (duh!):
- Setup a Subversion repository
- Make sure everyone on the team has their own username/pass to the repository
- Get everyone setup with a subversion client
- Create a development and live environment
- Have a one-click deployment setup for pushing code from dev to live
The problem is that it’s a pain in the ass to get these things setup, so most people (including us) don’t do it for every site they have. It’s tempting to just get the site live and start working on it. “We’ll deal with it later!” I said.
Now I know – it’s absolutely necessary (and we’re currently getting this set up for FOWA, Vitamin and our other websites).
Enjoy this article?
If you liked this article, feel free to re-tweet it to let others know. Thanks, we appreciate it :)
We love .net Magazine
We're big fans of .net so they've hooked us up with a rad deal: Save 50 percent on your subscription.
Simon Willison
# December 13, 2006 - 3:21 pm
I’ve been using Bazaar NG instead of Subversion for personal projects recently, principally because setting it up is a great deal less hassle (you type “bzr init” in the directory with your source code in it and you’re ready to go). So far I haven’t used it in place of Subversion for team projects but I’d recommend it for any situation where the setup time involved with Subversion is a barrier to using source control.
Kevin Cannon
# December 13, 2006 - 3:51 pm
Sounds good.
I’d be interested in hearing more about the dev site and one-click deployment tools. Would you be able to share any more info on that.
I’ve used SVN, but never had a huge need to go further.
James Deer
# December 13, 2006 - 4:03 pm
You make a good point there Ryan, just wondering if you know of any resources that would help people set up this type of environment (tutorials etc).
Robert
# December 13, 2006 - 4:16 pm
Hind-sight is 20-20. SVN is an absolute must for development and is great for just about anything that you are constantly updating.
James what do you want to learn: svn setup, a dev and production environment, etc?
Gordon Montgomery
# December 13, 2006 - 4:17 pm
I wonder what 37signals would have to say about you following what seems to be very similar to their ‘get real’ approach and then hitting this untimely but real wall … [Jason Fried's email removed by Ryan to avoid spam bots]
Ryan Carson
# December 13, 2006 - 4:30 pm
Sure Kevin. I’ll try to remember to post something about that soon.
Edward
# December 13, 2006 - 4:41 pm
What about a staging site? You mention pushing from dev to live, but you don’t want to go live until everything’s tested. As you have multiple developers, unless you have an intermediate staging level, you can’t test everything together without either publishing to the live site or freezing development.
Ryan Carson
# December 13, 2006 - 5:07 pm
In my experience, it’s not really necessary. Most of the development is done on the developers local machine/server and then pushed to Dev.
We have a staging environment for both DropSend and Amigo and we’ve never used it.
Ryan Carson
# December 13, 2006 - 5:14 pm
Nice tip Simon – cheers.
Curt Sampson
# December 13, 2006 - 6:14 pm
I find this news rather cheerful, in a bizarre way; it’s nice to see that someone else out there has the same problems we have, and may actually find useful some of the tools that we (at Starling Software) have built over the last year or two. This will encourage me to put something together and release it.
We’ve got a standard directory structure and system, using Subversion for version control, for quickly building, testing and deploying websites. The only tool we’re really tied to at the moment is Subversion (though we originally started on CVS), since our import tools are heavily dependent on it. (These are tools to do things such as “bring http://foo.net/release/foo-1.3.2.tar.gz into the vendor area of the repository,” and then “bring foo into the external source s for project X”.) Other than that, the system is reasonably agnostic; you pick your web server (we use Apache or lighttpd), your languages (we use Ruby, sometimes our clients make us use PHP), the particular versions of libraries you need, and then make your own dirs for whatever datatabase schemas, libraries and applications you’re building. It’s designed around a full automated install and test framework, which allows you to bring up a server on any machine quite quickly, whether it be a personal development, demo, staging or production server. One particularly nice thing is that, if you have dependencies outside of the source tree, doing a checkout and test on a new host will very quickly tell you what you need to install there for the app to work. (Though mostly we try to keep dependencies in the tree–we almost invariably use an in-tree web server, database driver, and so on.)
Typically, given a new web project, we have the a production server up and running by the end of the first day. It’s inevitably simple–perhaps just a single page serving a few interesting titbits from the database–but it’s there, and from that point onward it’s just a matter of spending a few minutes to release a new feature after someone codes and commits it.
What we’ve just recently got working well is the tool sharing system, so that when we add a feature to a framework tool in one project, we can bring it back into “framework central,” and then in other projects easily pull in those new features. Without some automated help with this kind of chore, we’d never have all of our projects keeping up reasonably well with the framework. It’s invaluable for allowing your framework to evolve fairly rapidly.
I ought to write a paper on this one day.
Michele
# December 13, 2006 - 7:20 pm
Using Rails make it very luring to use SVN, because then you can enjoy the greatness of Capistrano. That’s one of the reasons why we’re using SVN on both of our apps.
I’m not sure, but I think you can use Capistrano even if you’re developing in PHP. You might want to have a look at it.
Bill
# December 13, 2006 - 11:06 pm
So how’s it going with selling DropSend? No news on your blog for a while.
cedsav
# December 13, 2006 - 11:59 pm
I personally don’t like the idea of a one-click deployment. It makes it too easy to push an update without thinking and testing it through. I guess it’s less of a problem for a web site though (as opposed to a web app).
Robert
# December 15, 2006 - 6:27 am
Ryan: Unfortunately, you are/were using PHP for this app. ;)
Using tools like svn allows for growing and versioning of your work, so you have to use it – in opinion. So, to take the pain out, I’d write a shell script for the basic setup of your svn. That way all you have to do is run a file, pass some parameters, and sit back a let it do it for you. (assuming you are doing your svn at the prompt)
I don’t think enough attention is given to DRYing up the setup process of development. I get tired of doing some of the same basic tasks to get my development environment/app setup. So, I’ve been writing shell scripts to help out with this process so I write once and then just run the program there on after.
Development servers are simply a subdomain for me that are deployed by a capistrano recipe pulling from svn. When production comes, I deploy to the production server leaving the development server running because we are usually not stopping at versions 1.0, but continue to develop the application.
I don’t use any one-click development tools. As I said, I use a lot of shell scripting these days to take the pain out and DRY up the process.
I’ll be having an article going up on the 18th over at the “Ruby Advent Calendar”:http://www.rubyinside.com/advent2006/ that talks about a script I wrote for doing all my setup for an edge rails application.
Any questions about this, just shoot me an email, I’d be happy to help out.
RedVodkaJelly.com - Quick Iteration
# December 17, 2006 - 1:13 am
[...] Ryan Carson also recently wrote a post about planning your software and deployment process at the start of projects so that you can quickly and effortlessly launch changes without all the messing about normally involved in launching or updating a web site or application. I follow the majority of the points that Ryan lists, but could really do with adding the final touched so that any deployments can be done with relativity minimum effort. [...]
dylan
# December 18, 2006 - 11:14 pm
We do a lot of shell scripting/make to make our deployments easy.
For example, we have a script that will autodeploy a new svn/trac server instance, add default users and permissions, etc., as a single command line task.
We make extensive use of our staging environment for testing. But rather than pushing from staging to production, staging gives us a place to test with large fake data sets, but in an environment with the same replication/scaling architectures we have in production. Those kinds of bugs and issues are really difficult to reproduce on a local dev box. It also prevents the “it worked on my machine” issue, as we make sure that staging and production have identical configurations, etc.
So for both staging and production, it’s a single command line task to update the app from a specific svn revision. This approach helps us immensely, and on a personal level, I hate to ever do something twice, so it makes me much less worried about that.
topfunky
# January 9, 2007 - 11:57 pm
Even though Capistrano is written in Ruby, I’ve used it to deploy Perl and PHP apps with ease. Highly recommended.
Ian
# April 3, 2008 - 9:54 pm
For my new website I require a host that offers SVN and SSH and I can’t find a shared host that does. The only shared host I can find is dreamhost.
Rackspace wanted £300 per month for their entry level managed server which had SVN and SSH but this is out of our price range at the moment.
Does anybody know of any other hosts who offer a reliable service but are cheaper than the rackspace price above