What is involved in a site upgrade?

For all the non-nerds out there I thought it might be interesting to give a small insight into what was involved with the site upgrade that happened over the weekend.

This also might be useful for any other admins out there upgrading from Drupal 5->6 (D5 to D6). Sadly this is still a little tricky, but hopefully upgrades will become less painful as the product matures.

Although Drupal 6 has been around since Feb 2008 it's taken 16 months to upgrade NoBMoB to this version, and here's why...

Background

Ah, but before we start, a small explanation: Drupal is the Content Management System (CMS) that powers NoBMoB. For the interested, this site sits on an dedicated server built from an GeForce 8200 motherboard (aBit branded), AMD 4850e CPU, 4GB 800Mhz CL5 RAM and 2x200GB SATA drives (Seagate Barracuda 7200.7) in RAID 1. It and the router that connects it to the outside world (via a 2.5mbps link) are connected to an APC Back-UPS BF500. Although NoBMoB isn't the only site on here it is by far the largest (around 5GB of filesystem space + 200MB database). This machine served up 2.5m hits in Jun '09 and over 30GB of bandwidth.

Why Upgrade?

One of the key laws of computing is "if it ain't broke, don't fix it" so why bother upgrading? Well - while this law has some truth to it, I wanted to implement some more site features, and also take advantage of some niceties that come with the later version of Drupal. Sure, we could carry on using the old version, but then any development would be locked to that and could not be shared with the world. Drupal is an open source system after all and it's nice to give back. Look at the Trackfield project for instance - this was developed to show altitude graphs here has been released to the public so is out there in use on other similar sites around the world.

So Why Wait?

One of the great things about Drupal is that it's an open source, modular system.

One of the problems with Drupal is that it's an open source, modular system.

That is to say, it's great to be able to download and use someone else's modules for Drupal, but if they don't work on upgrading these to the next release then an entire site upgrade will not be possible. It doesn't help that the philosophy of Drupal development is, "out with the old, in with the new" which means various APIs change between major releases. While I'm not completely enamoured with this, you can see the point. Think how much better certain O/S and computer architectures would be if their hands weren't tied by backwards compatibility?

So the big wait was mostly down to the availability of various modules in use here. Even then there were problems and I had to adopt a couple of lesser used modules and create the D6 versions myself (thankfully mostly using patches that had been supplied by other people).

Process

So we went a little something like this:

• In one of the test environments (there are 3 development/test environments on the dedicated server, on top of the production site you're reading now) download and install Drupal 6.
• Install all the associated modules, including the adopted ones (out of interest there are 43 in use here!).
• Aside from the modules that are of some use to the public, there are numerous ones that only make sense on NoBMoB (things like showing the "who's in" list, creating ride galleries, etc). These had to be upgraded and there are 18 of those!
• Well - there are 18 NoBMoB specific modules now. Some of the various functions that these do were previously coded into the old Drupal 5 theme. This was originally done as it was just easier, but it makes upgrading hard as we couldn't bring the old theme along (yes, nice as it was I didn't choose to change the whole look of the site). So a lot of code was moved from theme functions to modules.
• Create a new 'nobmob_views' module. Views control all the lists you see about the site and there was no automated upgrade path for views. Because of this, after manually upgrading all the views used here they were exported and coded into a module so that when the upgrade happened for real all the views were 'just there'.
• Find a nice D6 theme that will work for us. This theme is actually based on Multiflex with several tweaks - including several image changes and quiet a bit of CSS. It may change more due to that annoying IE6 problem we've since discovered
• The major upgrade process in Drupal is supposed to be: turn off non-core modules, run upgrade script, turn on non-core modules. This is OK, but those turn on/off parts are annoying and time consuming. To get round this I wrote a couple of scripts to do this work that were executed at time of migration, and that could be run again and again to test it.
• Perform final tweaks. Like enable your new custom theme, set up various things that the usual upgrade doesn't do for you. As these are laborious another script was created to do this. Handy as the upgrade can be tested again and again until the previous scripts, and this one, are all working well.
• Do it for real! This happened last Friday night. Take the site offline (I actually point the virtual host at a completely blank 'offline' site rather than use Drupal's own function). Backup the entire site somewhere safe. Copy in all the stuff above (new Drupal 6 system, custom modules, etc, etc). Run the upgrade scripts. Hope it works as tested

And Viola!

Wrap Up

FWIW, when the modules were available this wasn't actually that hard. Things could be made a bit easier if the core Drupal upgrade script could be called from another one, but that's not a big deal. I only had three real steps to perform on go-live which could be cut to one if this were the case. It probably only took a week or so of testing upgrade scripts and process before doing it for real.

Of course, nothing is perfect and there are some lesser used things that are not yet available here. Subscriptions for example, and some admin things that normal site users won't even miss. However, they will return in the coming weeks if enough people request them or I notice enough bad requests for missing stuff in the log files. It's interesting to see what people do actually use I guess.

So there you have it - such is the life of a nerd. Enjoy!