Anyone know who’s responsible for this? I’d like to know where they get their data.

UPDATE: The WHOIS record unexpectedly contains useful info; email dispatched, never mind.

A few traumatic hours ago Ogged alerted me to some alarming allegations about my beloved Man Vs. Wild (known as Born Survivor in the UK):

To live up to his public image of a rugged, ex-SAS adventurer, it must have seemed essential for Bear Grylls to appear at ease sleeping rough and catching his own food in his television survival series.


But it has emerged that Grylls, 33, was enjoying a far more conventional form of comfort, retreating some nights from filming in mountains and on desert islands to nearby lodges and hotels.


Now Channel 4 has launched an investigation into whether Grylls, who has conquered Everest and the Arctic, deceived the public in his series Born Survivor.


…


[A]n adviser to Born Survivor has disclosed that at one location where the adventurer claimed to be a “real life Robin-son Crusoe” trapped on “a desert island”, he was actually on an outlying part of the Hawaiian archipelago and spent nights at a motel.


On another occasion in California’s Sierra Nevada mountains where he was filmed biting off the head of a snake for breakfast and struggling for survival “with just a water bottle, a cup and a flint for making fire”, he actually slept some nights with the crew in a lodge fitted with television and internet access. The Pines Resort at Bass Lake is advertised as “a cosy getaway for families” with blueberry pancakes for breakfast.


In one episode Grylls, son of the late Tory MP Sir Michael Grylls, was shown apparently building a Polynesian-style raft using only materials around him, including bamboo, hibiscus twine and palm leaves for a sail.


But according to Mark Weinert, an Oregon-based survival consultant brought in for the job, it was he who led the team that built the raft. It was then dismantled so that Grylls could be shown building it on camera.


In another episode viewers watched as Grylls tried to coax an apparently wild mustang into a lasso in the Sierra Nevada. “I’m in luck,” he told viewers, apparently coming across four wild horses grazing in a meadow. “A chance to use an old native American mode of transport comes my way. This is one of the few places in the whole of the US where horses still roam wild.”


In fact, Weinert said, the horses were not wild but were brought in by trailer from a nearby trekking station for the “choreographed” feature.

Josh has been disqualifieded amidst a cloud of scandal, Dumbledore’s dead and now Man Versus Wild is a fraud — where the hell are all the male role models going!?

Fortunately, a commenter on the story offers a ray of hope:

Bear is for sure coming after the person making the allegations and there is no escaping him, not even in the remotest parts of the world. He will hunt you down.

We’re going to fight this thing!

I haven’t started reading Deathly Hallows yet, although Emily and I did break down and sheepishly buy a copy from the customer service desk at Giant last night. She’s most of the way through, but is currently taking a break. I’ve been rereading a borrowed copy of the sixth book (thanks Matt!), reminding myself of important contextual points like who the hell Romilda Vane is and how to pluralize “horcrux”.

I’ve also found myself reading a lot of blog posts about whether Harry Potter is good or stupid or could beat up my dad. Most of these are merely the forces of curmudgeonliness checking in for another round of the fight that Michael Chabon started (it’s both a children’s book and a wild commercial success, so they like their odds).

There’s also the willfully counterintuitive posts — folks who decide not to let the books charm them, then write 300 words about how they don’t see what the big deal is. My favorite so far is Megan McArdle’s economic critique of Harry Potter, the subhead of which contains the hilarious assertion that “successful magical worlds depend on basic economic principles”.

It’s tempting to quibble with Megan’s complaints about the apparent limitlessness of magic, dismay at the authorial sloppiness implied by a reliance on imperfect communication between characters, and puzzlement over how stratification of wealth can exist in a society blessed with extraordinary abundance. But plausible answers to these criticisms can be found in the AD&D rule book (any edition), O Henry’s short stories and contemporary American society, respectively. So I won’t belabor the point.

I do think it’s interesting to consider the books’ appeal, though. Kriston provides a nice account of it here. But I think my own reasons come closer to Ogged’s: these books are hugely comforting. The reliable patterns of discovery, worry-prefaced resolution and compassionate authority figures are incredibly soothing. Plus there’s the handily straightforward good/evil dynamic, which makes even the tension reassuring in a Rocky IV-ish sort of way. More than anything, there’s a pervasive sense that everything will work out in the end. Or there is for me, anyway — perhaps those who’ve already finished the new book will disagree.

This is getting weird.

I like how some of the shots of “off-air” talent are screencaps from on-air appearances. And how some of the contestants don’t actually work in media at all. But the exclusion of certain perennial candidates is, of course, an outrage.

To get things started off nerdily on this beautiful Monday morning:

• Xbox-Scene translates an interview with TheSpecialist, the guy who’s responsible for the XB360 DVD firmware hack and is helping further the state of the art in attacks on the core system. Microsoft did an impressively good job on this console’s security: hackers have decrypted pretty much everything on the system, but still can’t run homebrew code. Yet.
• The RIAA is scared of Harvard. The article tries to make it sound as though the school’s big scary law professors are responsible. But I think we all know that it probably has more to do with secret convocations of robe-clad figures, who clutch ceremonial daggers while writing our nation’s energy policy and Simpsons episodes. You’re not fooling anyone, Yglesias.
• Remember when I was all excited about Pownce? Well, Rich was nice enough to send me an invite. I tried it and it kind of sucks. Twitter + file transfer would be great, but the client doesn’t integrate as cleanly with OS X as Twitterific does (i.e. there’s no Growl functionality) and whenever there’s a network error the whole thing grinds to a halt until you notice what’s happened and click a button. I was also disappointed to see how half-baked the event stuff is.
  
  
  But none of this is stopping Valleywag from wetting themselves over the service in a deeply embarrassing way, then whining that no, they were just kidding (or something) in the comments. Bah! For more, see Uncov.
• If you’re interested in Processing and not yet reading Flight 404, start. It’s written by one of the Barbarian Group guys, and he uses it to detail his experiments with artwork generated from physics simulations. His visualizations of magnetic fields are particularly nice, I think.
• Motors! Building a soda-can motor might be fun.
• Oh yeah! There’s this, too: a new version of the Google Analytics OS X dashboard widget. Nice. Thanks for the link, JP.

The exterminator’s aerosol poison arrived a day before my mail-ordered Terro, but it didn’t last. I consider this a very promising start:

I think Tim’s being just a bit disingenuous here when he discusses the relative competitiveness of the cell phone market:

It’s nonsensical to say there’s no competition because consumers only choose a wireless carrier once a year (or even once every two years). Most people don’t buy computers, cars, or major appliances more often than that, yet no one claims that makes those industries uncompetitive. If consumers get crappy service during their contracts, they remember this fact and switch to a different carrier at the end of the contract period. And consumers comparison shop before they sign a contract, so phone companies have as much incentive to keep their prices low in a contract-based system as they would in a system without contracts.

It’s true that purchasing at intervals doesn’t rule out competition. But this argument ignores a few salient points about this particular market.

Nearly all phones are locked, minimizing their resale value. Transferring your contact information between low-to-midrange phones is difficult or impossible. Until recently, you couldn’t take your phone number with you (number portability is a much-loved big-government mandate that the carriers fought tooth & nail). Although some carriers let you transfer your contract to someone else without paying an ETF, doing so while retaining your number is more than a little tricky, if not downright impossible. And of course, contracts typically last longer than a mobile phone’s window of technological relevance, ensuring that these barriers will be in place until the phone is worth only a fraction of its initial value, needs a new lithium battery, and is generally headed for the donation bin.

All of these factors inhibit the emergence of a viable market for used cellphones and plans, which in turn allows the carriers to avoid some of the sorts of competitive pressures facing refrigerator and automobile manufacturers. Yes, they have an incentive to keep their prices low in a contract-based system — but only within the bounds of that system. It’s as if NFL players collectively agreed to play touch football for a season and used their union’s power to freeze out replacement players. There could still be competition, but it wouldn’t be nearly as vicious as it would need to be to optimally serve the public.

There’s also this:

The iPhone point also strikes me as especially silly. The iPhone is expensive because it’s a cutting-edge gadget that’s been on the market less than a month. The fact that some of the cost comes in the form of a 2-year contract, as opposed to an up-front sticker price, is beside the point. If you think the iPhone, 2-year contract and all, is too expensive, buy a different phone. There are plenty to choose from.

But the point Ars is making is that the iPhone actually isn’t being subsidized by the contract fees. Consumers are buying the hardware at full retail price and being locked into a contract. This puts the lie to the carriers’ argument that early termination fees are in place to avoid losses over hardware subsidies — they charge the fees whether there’s a subsidy or not (and only one carrier will prorate this fee). Nor do customers have a way of opting out of this fraudulent system of portable phone paternalism. As Farhad Manjoo wrote yesterday:

You can’t offer to pay full price for a handset in exchange for a reduced early-termination fee and an unlocked phone — no major firm will let you do that. Indeed, in some instances carriers will charge you a fee even when they don’t offer you any break on the price of the phone. The iPhone is the primary exhibit: AT&T doesn’t subsidize the price of the phone, but you’re still locked to a single provider.

I’m currently facing an ETF from T-Mobile for an account I set up without receiving a handset (I used it for my now-defunct DCist SMS service). In retrospect I should’ve signed up with a prepaid service. That would’ve been fine for my application, but these services typically charge more for services and offer less advanced phones. They’re designed for customers who are credit risks (or drug dealers!). Unlike in Europe, prepaid is clearly not a viable way to opt out of the contract system.

The actual reason the fees are in place is to prevent “churn” — the frequent switching between carriers by customers seeking better service, handsets or prices. Obviously this is a nuisance to carriers, and has real administrative costs. But surely the overhead to customer acquisition and early account termination is less than $150 (and if it is, surely it ought to be pro-rated when a customer has fulfilled part of their contract). No, the carriers are transparently using the fee to introduce friction into the marketplace, allowing them to extend the lifecycle of each generation of technology and spend more time in the pleasantly wide part of the margin that comes once they’ve paid off their capital investments.

There’s nothing wrong with wanting to post nice fat profits, but I don’t see a reason why we should enable this cartel-like activity. Customers will be better served by a faster-moving market. As it stands, customer dissatisfaction remains high across the industry, text messaging rates are going up rather than down and all customers with sub-smartphones remain locked into buying software exclusively from their carriers, despite countless forums filled with people vainly asking how to load mp3s as ringtones. And although the internet jumped all over Ted Stevens for wanting a phone that would seamlessly rollover from wifi to cellular to landline access, his request wasn’t unreasonable (well, except perhaps for the bit about the motorcycle) — VoIP/cell handing-off is useful technology that’s been available for years, but is just barely showing up in the consumer marketplace.

It’s simply not a very good or competitive market. Everybody hates it, and we ought to fix it. I have no idea whether a more open market would increase the total monthly expense of owning a cell phone. But even if it does, I suspect it would open up a lot of potential for growth of other industries — independent cellphone software, least-cost VoIP routing, third-party ringtones, better handsets, location-based services and who knows what else. Wireless data is a utility, not a retail product. Its market ought to be opened up.

Thanks for the reminder, Ezra! Man, I can’t believe it’s been a whole year since MediaBistro’s last DC journo beauty pageant. As you may recall, last year’s contest featured a stunning come-from-behind victory — one that unfortunately didn’t count due to the deciding surge of democratic expression occurring in the 24 hours after voting closed. This in turn can be attributed to the parties responsible for the outpouring of support accidentally misreading the end date of the contest. I bet they felt pretty stupid about that.

But don’t worry! I’m confident that democracy will triumph this year.

Adrian Holovaty has just released a Python library called templatemaker that promises to make screen-scraping much easier.

Some explanation: if you’re trying to do something automatically with a website, it’s best if it has an API. This is a reliable, documented interface that your program can use to interact with the other site. Flickr has an API; Google has an API; del.icio.us has an API. Lots of sites have APIs.

But lots of sites don’t. WMATA, for example, has none. In that situation you’re stuck writing a program to parse the HTML of the site, which was designed to be viewed by humans, not robots. Here’s a very simple made-up example. This table:

Looks like so in HTML:

<table align=”center” border=”1″ width=”300″>

<tr><th>Time</th><th>Line</th></tr>

<tr><td>9:00 AM</td><td>Orange</td></tr>

<tr><td>1:00 PM</td><td>Blue</td></tr>

</table>

And I can write some regular expression-laden Perl (or whatever) to extract info from it like so:

while(my $l=<>)

{

   if($l =~ m/<tr><td>(.*?)<\/td><td>(.*?)<\/td><\/tr>/i)

   {

      print “train time is $1 and color is $2\n”;

   }

}

This is a pain in the ass. Worse, if WMATA decides they want to make some minor cosmetic changes — altering the background color of each odd-numbered result row, or changing to 24-hour time notation, or even just adding some line breaks to make their HTML more readable — the regular expression will break and have to be meticulously redesigned. And this is just a simple example; in reality the initial regex will be much more unwieldy and the fixes much harder to identify and make.

Holovaty’s templatemaker aims to automate this by matching similar stretches of text, allowing it to determine which portions of a set of documents vary and which remain constant, then plucking the variable data out on the basis of its analysis. Some of that will be junk — you probably aren’t actually interested in the breadcrumbs or meta tags in a set of pages, although they’re quite likely to vary across the set — but with an intelligent look at the results, this could greatly speed up the process of extracting data.

Check out the sample usage for a simple example of what I’m talking about. I’m not much of a Python-lover, but I’m looking forward to using this project.

Walking into the Old Dominion Brewhouse yesterday, two guys stopped me and asked if I could direct them to The Space. Clearly I have no idea what the market actually wants.

In other news, I’m interested to hear if there’s a section on La/te Ni/ght Sh/ots where users can employ any HTML — links, images, whatever. If you’ve used the site enough to notice such a spot, let me know. Science projects are hatching.