Alex Payne points to a blog post by Ben Laurie that discusses Diaspora and Haystack, and how projects like these can attract huge amounts of press, only to flame out as their charismatic founders’ incompetence is revealed.

I agree with Ben’s post, but it’s worth being a bit more explicit about what allows these situations to arise: the quality of most tech journalism is abysmal. I mean really inexcusably bad. Mainstream publications regularly assign writers to cover the software industry that have a level of understanding regarding the field that would be unacceptable in an intern. The most esteemed practitioners in the tech press are either focused on the consumer electronic user experience or are building personal brands around faith-based tech triumphalist movements.

In this sort of environment, it should be no surprise that an embarrassing hype cycle can emerge — one that talented self-promoters will use to enhance their status and wealth. I find it difficult to assign all that much blame to those self-promoters: the whole problem is that they don’t know any better. What more can we expect? Besides, it’s very easy to start believing your own bullshit once people with seemingly-meaningful professional credentials start validating it. Self-promoters will self promote; it’s not realistic to expect them to be the ones providing diligence.

I suspect that the problem may have to do with the structure of the industry: if you know much about it, you’re probably going to be able to make more money participating in it than writing about it. I don’t know enough about finance to really judge, but it seems as though that press sector suffers from a similar systemic disability — certainly all can agree that the financial press didn’t cover itself in glory in advance of the recent financial crisis. Once that story became big enough, talented generalist journalists filtered in and did the job properly.

But unless and until the skill premium for the software industry diminishes relative to journalism I’m not sure there’s a good way to align incentives in a way that fixes this problem. The best we can do is to recognize that the journalists who wrote excitedly about Haystack and Diaspora made a mistake; they were fooled, and they wasted our time. There’s no need to tar and feather anyone, but their credibility needs to suffer if we want this situation to improve.

Maybe we don’t need it to improve! It’s not that important, to be perfectly honest. But it sure does bug the hell out of me.

I’m taking some grim satisfaction in the dissolution of the Haystack Project. If people have actually been hurt by this software, I’ll be removing the “satisfaction” part of that reaction.  But given the relentless overhyping of technological interventions in pro-democratic organizing, I’m hopeful that little actual damage has been done.  These technologies will no doubt soon become genuinely crucial to people living under repressive governments, but I’m optimistic that Heap’s work didn’t spread far enough to cause serious trouble.

A few key links:

• The EFF says to stop using Haystack
• The resignation letter of its main developer
• Ed Felten’s take
• Nancy Scola discusses the media’s inability to report responsibly on technology (and is very nice to me, though I hasten to say that I’m no security expert)
• The twitter stream of @ioerror, beginning at this tweet

I think Nancy’s piece raises the most important questions: why did the media — and more importantly, the federal government — fall for Austin Heap’s bullshit?  Here I disagree with Nancy: I don’t think it’s just a question of technical expertise.  I’ve been skeptical of Haystack since its announcement, but I couldn’t write a secure crypto proxy to save my life.  That stuff is really hard.

But here’s the thing: the technology is hard, but it’s also done.  Smart people have looked at these problems and have solved the ones that are solvable.  There’s work left to be done at the edges — better hash algorithms, that kind of thing — but I guarantee that’s not the sort of flaw that’s afflicting Haystack.

The thing to realize is that good security is a set of patterns.  Relatively few of them have to do with technology (although learning about the technology behind security systems can certainly help you understand the patterns).  The media’s naivete springs from the idea that the right app could liberalize Iran.  Ignorance of the app’s internals is a much smaller problem than the sort of wishful thinking that gets that story pitched and approved (and read).  Recognizing a snake oil salesman is just as important as recognizing an exploit.

Alarms should have gone off when Heap announced he was writing new software.  Why write software?  We have software.  Did Heap think the limitations of Tor and Freenet and Freegate and PGP and all the others exist because those tools’ authors are lazy or stupid?  That was never the problem.

I guess what I’m trying to say is that reporters should spend more time reading Bruce Schneier.  Even the repetitive entries. Especially the repetitive entries.