yelvington.com

Every one of us who opens up a website to public conversation faces a daily annoyance: blog and comment spammers. Cleaning up after them becomes a morning ritual.

Google, a mix of scammers and legitimate businesses, and global economic inequities all play a role in creating a system that guarantees this problem will not go away any time soon.

The result is a mix of overt and covert spam. Overt spam is easy to spot; it's usually just a set of links to e-commerce sites, often peddling fake merchandise. Covert spam is disguised. Here's an example:

Several characteristics stand out:

1. It's not from a bot or a script, but was posted by a human. Registration systems and CAPTCHA challenges are pretty good at stopping the bots. After you take those measures, you're left with posts like this.
2. It's awkward. English is apparently not the writer's native tongue.
3. It's generic, not related to the content. This can be tricky, because the manual spammers have learned to cut and paste phrases from content and from other comments.
4. The links point to a commercial website. This is where Google comes into play. The point of this spam isn't to get your users to click. The point is to steal the reputation of your site. Google's algorithm values inbound links, especially if they are from "respected" websites that already have good Google rankings. These vultures are gaming the system to boost their Google search rankings.
5. It was posted at an odd early-morning hour, a tip that if you do a bit of detective work with your logfile info, you'll probably trace the posting back to India, Turkey, Romania, Russia or China.

So how does this happen? This morning I ran across a bit of spam in which a spam factory was looking to hire spammers:

I tracked these guys down. It's an IT company in India, offering to pay 5 Indian rupees -- about 11 cents in US money -- for every "ad" posted on "free classifieds" sites around the world.

But it's not just Craigslist that they're targeting; in their Frequently Asked Questions section they admit that forums and blogs are the target, too:

And they're offering a vague "special bonus" if the ads are posted "in foreign countries:"

I did a little math. The ad claims a range of 10,000 to 25,000 INR per month; that's $220 to $550. The website of this company claims up to 45,000 INR, which is about $1,000.

India has people riding around in Bentleys and Rolls-Royces, and I saw a lot of Rover and Toyota SUVs on my visit. But it also has millions living in brutal poverty and millions more barely above that.

Let's say you're a highly motivated, English-speaking college student with a few basic computer skills. How many spam ads would you have to post to live at that magical 45,000-rupee level? With a 40-hour work week, it would be nearly one ad every sixty seconds. Hard work, actually. But there are millions undoubtedly ready to try it, and if the spam factory simply fails to deliver the promised money order, there are millions more ready to try next week.

Similar conditions exist in Romania, the former Soviet countries, China and many other places around the globe.

The cash that feeds this global spam engine comes from a number of sources. In 2003, when the US housing bubble was running full steam and mortgage bankers were lining up the bad loans that set up our current economic recession, MSNBC followed the money trail of some e-mail spammers and found that it led back to "big-name companies like Ameriquest, Quicken Loans, and LoanWeb." Mortgage lead-generation fees and commissions were ultimately funding the trash in your inbox.

But there are other, more shady sources, ranging from U.S. companies selling term papers to lazy students willing to cheat, to scammers selling counterfeit Rolex watches and fake prescription drugs, technothieves pushing spyware onto your PC, and pump-and-dump stock market frauds.

The CAN-SPAM act gave the US government some tools for cracking down on email spam, and people like Alan Ralsky have been sent to prison. But the law doesn't apply to Web sites and social networks. And while the US government isn't shy about sticking its nose into other countries' business when oil is involved, you'd better not hold your breath waiting for it do something about offshore spamming.

For years I've been pushing the idea of participatory journalism. If you've been thinking this is just theory, some more data has emerged that ought to get your attention:

• "According to Web measurement firm Compete Inc., Facebook has passed search-engine giant Google to become the top source for traffic to major portals like Yahoo and MSN, and is among the leaders for other types of sites." (San Francisco Chronicle)
• "Facebook was the ­#4 source of visits to News and Media sites last week, after Google, Yahoo! and msn. ... Facebook could be a major disruptor to the News and Media category." (Hitwise analyst Heather Hopkins)
• "Of the 71% of the adult population who get news online, 75% of them say they get news forwarded to them through email or posts on social networking sites. ... In addition 23% of the social networking users who get news online say they specifically get news from news organizations and individual journalists they follow in the social networking space. " (Pew Internet & American Life Project report, "Understanding the Participatory News Consumer")

News is no longer just a report. It's a conversation, a broad process in which many people contribute to varying degrees.

This is not new, but rather a restoration of normalcy that was disrupted by Gutenberg and Marconi. We humans evolved in tribes, and we digest news and information by sharing it and talking about it with our friends and family. The one-way flow that is characteristic of print and electronic broadcasting is at odds with our nature. The Internet ends that directional tyranny.

But that's only part of the change. Social filtering and recommendation are the new editors and publishers. Years ago, I said that "gatekeeping" was dead and that we would need "guides" to show us the way through the chaos of the new marketplace. Our friends and followers are now taking a prominent place in that "guide" process, displacing the "editor" and the "newspaper" product.

Let's start with a confession: I don't like JavaScript. I don't like object notation and I don't like programming languages where whitespace (line enders) is significant. I cut my teeth on C, and I am suspicious of any deviation from its spartan truth. I also don't trust power windows and think the Volvo 240 was the pinnacle of automotive engineering, just to put it all in context.

And I already have a book on JavaScript: "Programming JavaScript for Netscape 2.0." It's on the dusty, rarely visited end of my bookshelf, right next to "Internet Starter Kit" with a floppy disk containing TurboGopher 1.07.

So I'm not a great candidate to review the book on JavaScript that Packt shipped me a couple of months ago.

Nevertheless, I'm impressed with Matt Butcher's "Drupal 6 JavaScript and jQuery."

Like it or not, the poorly named JavaScript (it has nothing to do with Java) is the most dependable client-side programming tool we have. With Apple's recent attack on Flash, which is banned from the iPad, it's going to become even more important. If you want pages to do anything more than sit there and look at you, JavaScript is your tool, period. So deal with it.

Butcher's book is really about three things: The JavaScript language as it's currently implemented, the jQuery library that gives JavaScript a set of simple but powerful incantations that can cast magic spells on HTML, and how both of them fit into Drupal's modular, themeable framework.

In the preface, Butcher declares, "you will learn everything you need in this book." That's overly ambitious but only slightly so; if you're generally familiar with HTML and pretty much any programming language, you'll be able to follow along without difficulty. Butcher is literate (he's working on a Ph.D. in philosophy at Loyola), a veteran technical writer (five books for Packt Publishing) and a bona fide Drupal developer (for Chicago-based Palantir.net). He knows his stuff and explains it well.

If you're a designer working only in Photoshop and you've never written a line of code, you're going to have a tough row to hoe, but even then you can probably keep up if you put in the energy. Butcher assumes little and explains everything, but does so efficiently. If you need to be told three times, you're going to have to read three times.

As the book moves along, it progresses from being heavy on explanation with brief examples to being heavy on exercises with brief explanations. By Chapter 3 you're extracting data from the DOM, manipulating CSS, and creating content carousels. By Chapter 4 you're writing a text editor, a project you'll keep enhancing later in the course. By Chapter 7 you're fetching structured data across the network (AJAX, JASON, AHAH, etc.) By Chapter 8 you're writing Drupal modules and qualifying for your wizard's hat and gown.

The jQuery library is an open-source project with many benefits, including hiding some of the occasionally ugly differences between the ways two browsers behave. It's been adopted as an official part of Drupal, but also has the backing of Nokia and Microsoft and shows up in many other Web applications.

When it's properly used, it can greatly improve Web usability. I've been looking at Drupal 7, which is not yet released, and it's completely transformed the experience of managing content in a Drupal environment through smart application of jQuery-driven effects. When you want to edit something in D7, you just point at it, click the control that magically materializes, and watch as an editable version appears in a layer, courtesy of jQuery, AJAX and standards-compliant HTML.

I've been quite frustrated this year with JavaScript as we've struggled to improve page-rendering performance on our newspaper websites. Third-party JavaScript from various advertising engines and video services has been killing us. If everyone used jQuery, we'd all be better off. The core library is tight and highly compressed, and it makes possible an awesome economy of expression. Because it's so well supported by third-party extensions and plugins, we've been able to pretty much banish Flash from our site content for everything except video delivery.

If you're a developer and you aren't familiar with these tools, you need to be. And I'd recommend Butcher's book as one of the best-crafted technical training manuals I've come across. Dig in.

Life is change, and we've had some great people change their lives by leaving Morris DigitalWorks to take on new challenges in the Web consulting and development world. We're sad to see them go, but excited when they wind up working on cool projects like Whitehouse.gov.

So we're looking to grow a new crop of wizards, and in the mix we're going to be recruiting some journogeeks.

A journogeek is someone who loves Web technology, is a creative problem-solver, and has an aptitude for growth but also brings a commitment to our journalistic mission.

We've made a commitment to change how we work. We're placing Web tools in the hands of every journalist at every one of our newspapers. These tools need to be made better, and that requires close collaboration among journalists and developers, and iterative development.

We've made a commitment to the principles of Open Source as well, not only using it but contributing significantly to the Drupal development community.

If this sounds interesting, feel free to contact me and I'll make sure you're notified of each position as it comes available. Right now we're looking for a senior designer. Upcoming shortly we anticipate several developer positions that might range from intern up to grand wizard. Bring your own hat.

It hasn't been a good month for Microsoft. First Google with its Nexus One, then Apple with its iPad, have highlighted how its empire is in risk of falling, replaced by a new mobile world in which Microsoft is irrelevant.

Most revolutions fail because the revolutionaries can't stay united. This one is no different. And there is plenty of skirmishing among the revolutionaries.

Apple and Adobe are engaged in a battle in which Apple seeks to use HTML 5 to snuff out Flash. Google and Apple, once lovey-dovey to the point that Google's CEO was on Apple's board of directors, are squared off over Android. And so on.

But Microsoft continues to be the big target. Specifically, it's the Microsoft-dominated "desktop model" of computing.

I'm not talking about the size or portability of the PC, but rather the metaphor of a desktop, folders, and applications. It's a world in which the so-called network effect makes it nearly impossible for any competitor to catch up. Regardless of how much money Apple spends on those Mac vs. PC commercials, the Mac will forever be a minority player.

So both Apple and Google -- for very different reasons -- are setting out to smash the empire by creating entirely new systems.

The iPad aims to create a new model of consumer-focused computing and rally independent developers into an army totally controlled by Apple, unable to function without dependence on its company store. That army's rules are harsh. If you wonder, go read the terms to which you have to agree in order to become blessed as an iPad developer. The reward is a perhaps better life than independent developers have had under the bitter rule of Microsoft, which has a habit of visiting upstart startups with an offer they can't refuse.

Google's aims are different: to perpetuate its vision of an empire built by organizing all the world's information, not by creating sexy consumer devices. Microsoft is the single biggest threat to that vision, so Google has set forth two efforts that ultimately aim at unseating Microsoft (and laying a moat to keep Apple away).

One is Android, which is clearly modeled on the iPhone/iPad system but in a far more open way, free of secrets and not-so-veiled threats. The other is Chrome -- the operating system, not just the browser.

I test-drove ChromiumOS yesterday just to understand its potential. I downloaded and installed it on a 2-gig thumb drive, stuck it into my semi-netbook (11.6-inch Acer), and rebooted.

It's blazingly, in fact shockingly fast. In a matter of seconds I could log in (booting Windows would take about 90 seconds). I typed a username and password and instantly I was running the now-familiar Chrome browser, pointed at a page displaying icons for popular services like Pandora, Lala,Hulu, YouTube, Facebook, Twitter, Google Books and of course Google's Web apps such as Gmail and Google Docs.

There is no desktop and no software to install. It's all browser, all the time, running totally off my 2-gig thumb drive, ignoring my laptop's hard drive. This is clearly aimed at enabling devices under $200, and maybe even under $100, that have no moving parts.

But no desktop? No files and folders? No local applications to install? Then I have to have an Internet connection to use this, right? This is where HTML 5's support of a local storage database comes in, along with Google Gears. They make it possible for Web pages to persist and Web applications to continue working offline. ChromiumOS has a lot of work to do on that front, but the foundation has been laid.

Like Android, there's a security-hardened version of Linux underneath. Linux is industrial-strength software, powering all of Google's giant search and advertising services, but it runs on tiny systems, too. Apple's Darwin core, which underlies OSX and iPhone alike, is similar, if less mature. Their software isn't trapped on hardware whose specifications are dictated by Microsoft and Intel. It can be smaller, cheaper, taking tiny sips of power from batteries that might last days on a single charge.

At the Consumer Electronics Show this month, dozens of prototype tablets and netbooks were being shown. Chinese factories that today might make portable music or video players or GPS systems are gearing up to make Android or Chrome-powered Web devices. They're aiming low and they're aiming cheap, in the failure zone where disruptive innovation happens.

Clearly change is in the wind. Whether it's good or bad for us depends on how we respond. Picking sides is dangerous. As content providers or as software developers, we're probably smart to find ways to sell ammunition and supplies to all of the major opposing camps, if we can avoid getting shot in the process.

It's here. And I'm disappointed. It's not just that the iPad failed to live up to its hype (which was just short of ending world hunger, curing disease and raising the dead). It's that the iPad doesn't change the world, no matter how many times Steve Jobs says "advanced," "revolutionary," "magical" and "unbelievable."

It's certainly no savior for newspapers. What are you going to do, kill your website and sell your "publication" in the App Store? Nonsense. The iPad doesn't change the economic equation. You aren't prevented from selling your content by lack of technology and tools; you're prevented by a lack of market demand. And the demand isn't there because people have, at their fingertips, far more alternatives than the human brain can process -- literally billions. The iPad doesn't change that. If anything, it makes it worse by furthering mobile access to the Web. And by the way: This is 2010. If you're still thinking you run a "publication," you're dead already.

It's certainly not a new form factor. Computer makers have been trying to peddle tablets for years. They fall into a chasm between usable sizes. Palm engineered the original Palm Pilot by whittling wood blocks that would fit into a shirt pocket. Guess what: That's pretty much the size of the iPhone, Blackberry, Droid and Pre. Then you have the laptop, which has become the standard computer workstation. Shrink it too much, and it fails (7-inch netbooks are so dead, 9-inch netbooks are struggling). So what happens when you grow the iPhone into that size and shape? We'll see.

The list of shortcomings is stunning.

No multitasking. Are you kidding me? Palm Pre and Android have it, and they're "just" smartphones.

No Flash, so most of the video on the Internet won't play. Also rich-media ads won't play, so stick that in your "savior of journalism" pipe and smoke it.

No USB ports, so you can't plug anything in ... not your camera, not your printer, not your tetherable Blackberry that might give you Internet access. OK, you can fix at least some of that with an accessory that costs more, but Apple hasn't divulged the pricing. Maybe it can network over USB. Maybe it can't.

No 3G on any network other than AT&T. Can that be right? It doesn't support standard SIM cards, and if you manage to get a T-Mobile micro-SIM card, it still won't work because it's not on the right frequencies. Oh, by the way, if you're in Europe, get lost.

No camera? No Skype video? Whatever were you thinking?

Sorry, Apple fans, but I just don't see any innovation here.

Recently I've been on a downsizing binge, so I get the "small is beautiful" thing. For work, I traded in my Macbook Pro for a smaller 13-inch Macbook, and lately I'm locking it away at the office rather than carrying it around. Several times on international trips -- France, Germany, Spain, India -- I've traveled with my tiny Nokia N800 Internet Tablet and a folding keyboard. But even at a price (as low as $499) that's much lower than many had predicted, the iPad just doesn't seem to be fixing a problem.

Not long ago I bought a semi-netbook, an Acer laptop with an 11.6-inch screen. It's all the things that the iPad isn't. It's a real computer running real software. I can edit photos and even video (not great for that, but it can be done). Full-size keyboard, 160 gigs of hard drive storage, camera, Flash video, dual-core CPU, built-in reader for SD/MMC/XD/MemoryStick, Ethernet and SVGA projector output -- all of which the iPad doesn't have. It cost $100 less than the cheapest stripped iPad. Nope, I'm not tempted.

The new Augusta Chronicle website is now live, the latest in a series of conversions of Morris websites to a Drupal-based system.

Before and after

As I mentioned last week, there's a companion mobile-optimized site for smartphones, and if the browser detection is working properly, you'll go to http://m.chronicle.augusta.com/latest-news/2010-01-27/apple-introduces-s... automatically if you request http://chronicle.augusta.com/latest-news/2010-01-27/apple-introduces-sin... with a Blackberry, iPhone, Droid, et cetera.

But it's not just about technology. The reason we're moving to Drupal is to create an interactive environment where members of the community and everyone in the newsroom can engage in a broad civic (and, we hope, civil) conversation and contribute content in appropriate ways. In the old world, the Web was a specialty and most reporters and editors had limited access to the tools; in the new world, any staffer and any member of the community can post directly into the system with appropriate access controls.

I've written previously about the importance of context in evolving a journalism that conveys meaning to time-pressed citizens. One of the tools we'll be using for that is topics pages that frankly are modeled on those of the New York Times: a synoptic overview of the topic, links to incremental news coverage of the topic, and links to related Web resources. In an early brainstorming meeting with the Chronicle's newsroom, an entire wall was quickly covered with ideas for topics trackers. The Godfather of Soul is a well-executed example.

The Chronicle has one of the more active commenting communities among local newspapers I see. It's not always a pretty sight, but executive editor Alan English @aenglish09 has declared that comments will be "assertively moderated."

As with previous rollouts of the Morris Site Management System, this one comes with user profile pages

Major Drupal modules used in this project are predictable: CCK, Views, Panels, Imagecache, and so forth, plus a few that are new to us, including Context. We've written a few as well that may find their way into the Drupal contrib collection.