In the war room

Submitted by yelvington on October 9, 2008 - 11:08am

This week I'm in Jacksonville, Fla., where a team (right) is hard at work rebuilding Jacksonville.com on our new Drupal-based site management system. There are others up in Augusta and other locations, working as part of a larger virtual team, but even with instant messaging and regular conference calls there's no substitute for shoving a bunch of folks into one room with a sack full of junk food and not letting them out.

The Florida Times-Union's site will be the first on the new platform, followed by the Topeka Capital-Journal and the Conway (Ark.) Log Cabin Democrat. Launch has been pushed back a couple of weeks because of the elections -- we have high confidence in the hardware and software, but people will be stressed out enough without the added complexity of new tools on Nov. 4.

We're relying heavily on some Drupal contributed modules, especially Views (which lets you query the database and create various types of lists without writing SQL), the Content Construction Kit (arbitrarily structured special content types), FeedAPI (RSS and Atom acquisition), and Panels (arbitrary custom page layouts).

The result should be a system that lets reporters report, writers write, and editors edit without having to know anything about HTML, scripting, FTP and other online technobabble. Nevertheless, there are going to be some interesting training challenges as we move from a world in which the Web was the exclusive concern of a small team of specialists, to a world in which every member of the news organization can directly play an appropriate role in Web operations.

Comments

Submitted by James (not verified) on October 10, 2008 - 12:39am
Sorry about that. Drupal is such a turd. I ran two drupal sites for a couple of years and they were always broken. Never quite the polished look you expected. Then I installed an update to one of my modules and it crashed the whole site. It was completely unrepairable and the support staff were clueless. That's when I abandoned it for wordpress. Much nicer and reliable. Now I hear all sorts of horror stories from a friend who works for a webhost about drupal. Looking back, I'm glad I ditched it. In the 2 major revs and about 12 minor revs I used during that time, they never fixed glaring bugs and module updates would cause catastrophic failures. The 6.1.x module I upgraded near the end is what caused the site failure. Not even disabling it in the configuration table in the sql database would turn it off. mighty strange. It always seemed like a tinkerer's paradise and an editor's nightmare. It simply required too much work to produce professional results. But, I'm sure you have plenty of drupal experts on staff. Good luck with it.

Submitted by yelvington on October 10, 2008 - 7:22am
We've been running Drupal on newspaper sites since 2005 and never encountered anything like what you describe. But then, we're running on adequate hardware, not some $4.95 shared server. Drupal is not low-end blogware and should not be run on low-end resources. The look of a Drupal site is exactly as polished as you make it; the theming system is extremely flexible. If your site looks bad, blame your own HTML skills. As for your problems with contributed modules: You're supposed to make a backup and adequately test new modules before rolling them into production. Contributed modules are not part of the core Drupal distribution, are not guaranteed, and it's your responsibility to ensure that what you're using doesn't melt your server. If you need support, go to Acquia. If all you need is a very simple blog, go to Wordpress.

Submitted by Dave K (not verified) on October 14, 2008 - 2:56pm
The picture shows a bunch of folks working. How many developers and how many designers do you have working on this project? Also do you have a DBA? Not sure if you fleshed out your team in an earlier post. Sorry if this is somewhere already.

Submitted by yelvington on October 14, 2008 - 4:05pm
We actually had two people work on the design -- the AME/graphics for the Times Union (who established the basic look and feel) and a Morris DigitalWorks Web designer (who worked out a lot of the details). All of it has to fit into underlying geometries that had already been wireframed by MDW. I believe we outsourced the conversion from PSD to basic HTML (which took a couple of days). Adapting the HTML into Drupal templates is an iterative process, and several people have been working on that part, off and on, for a couple of weeks. The top-level stuff is really fast and the devil is in all the detailed formatting, CSS tweaking, etc. Most of the work that's been going on isn't software development in the classical sense, although we have written a couple of interesting Drupal modules. The guys you see in the picture are mostly focused on building Panels pages containing blocks of Views, which is a potentially tedious process but not one that requires lot of coding. It's mostly a matter of making a thousand small decisions and filling out a lot of forms. We do have a DBA, but he's not very involved in this project. He's had a hand in planning the replicated MySQL server array that will be running in the background. I haven't written about the hardware. My understanding is that we're configuring three layers. The first is a Squid proxy that will hide some details about the server farm (transparently rerouting requests for legacy documents to an appropriate server, for example). The second is the Drupal application servers, which will be redundant. The third layer is an array of replicated database servers. The net result should be that all of our old URLs should still deliver content, the Drupal system should serve pages instantaneously, we should be well-protected against hardware failures, and it should be simple and straightforward to apply more hardware to scale the system. We do intend to run many sites on the common hardware, which is the pattern Morris has followed in the past. This scale gives us some advantages when dealing with transient traffic spikes. I think we're going to be using memcached to move some of the Drupal cache functionality into RAM, which should be a great accelerator. Not clear on the bytecache, but we've used APC in the past.

Submitted by albert (not verified) on October 20, 2008 - 9:45am
Which version of Drupal are you installing now? From what I've played around with, not all contributed moduels are ready for 6, but 5 is getting to the point where a site running on it will be wanting an upgrade before long. Thoughts?

Submitted by yelvington on October 20, 2008 - 10:42am
We're building on 5.x because a number of modules we consider critically important aren't yet available for 6. We're eager to upgrade, though.

Submitted by Jordan (not verified) on November 12, 2008 - 9:06pm
Are you guys using the boost module? I worked on pilotonline.com and found boost to be helpful. Like you we relied on Memcache and a couple of other techniques.

Submitted by yelvington on November 13, 2008 - 9:04am
I don't think it's even been examined. Probably should be. We created our own "flattener" for some key pages when we rebuilt SavannahNow.com.

Submitted by Paroshep (not verified) on January 4, 2009 - 11:49am
Thanks for the photo. It drives home why my one man site can't compete with the professionals. Cheers,