The folks at the Sk/oll W/orld Forum were nice enough to let me write some rambling thoughts about “Big Data” for publication in, I’m guessing, a glossy brochure that will be flipped through if any of their event’s panels prove to be particularly tedious. I am grateful for the opportunity, and congratulate myself for not making any of the childish chewing tobacco jokes that were ripping through my mind during the process.

Normally I would sheepishly self-promote on Twitter and leave it at that. But it is maybe worth sharing how the editorial process went.  I originally wrote a piece that said, “You know, this stuff isn’t really new–the analysis of quantitative data and its application to social policy are at least a century old, and we probably shouldn’t expect any sudden home runs. But it is true that some new things are being measured lately, so who knows, we might figure some new stuff out.”

This was totally inverted, though, to emphasize the exciting new types of data and to minimize my yawning ho-hummery about the limits of technocratic governance. And it’s true that this inversion makes for a much more compelling piece! The editors I worked with did their jobs well (though I’ll admit I have no idea what the hed has to do with the piece).

But consider this datapoint when you read things about Big Data, or really any technology. Unless you achieve Morozovian levels of learnedness and vituperation, you’re not going to place anything that amounts to “Don’t Believe the Hype.”

…or one of them, anyway, was released today. You can find it here; or use this, a direct link to the PDF.

It really is unfinished — chapters end abruptly, discussions are alluded to that were clearly never written — and the whole thing is written (but not formatted) in Markdown. But it contains an extremely high percentage of Correct Opinions About The Web. Here’s one bit I particularly enjoyed about the Semantic Web (seemingly since relabeled as Linked Data, basically in order to escape this well-earned critique):

I have to say, however, the idea’s proponents do not escape culpability for these utopian perceptions. Many of them have gone around talking about the “Semantic Web” in which our computers would finally be capable of “machine understanding.” Such a framing (among other factors) has attracted refugees from the struggling world of artificial intelligence, who have taken it as another opportunity to promote their life’s work.

Instead of the “let’s just build something that works” attitude that made the Web (and the Internet) such a roaring success, they brought the formalizing mindset of mathematicians and the institutional structures of academics and defense contractors. They formed committees to form working groups to write drafts of ontologies that carefully listed (in 100-page Word documents) all possible things in the universe and the various properties they could have, and they spent hours in Talmudic debates over whether a washing machine was a kitchen appliance or a household cleaning device.

With them has come academic research and government grants and corporate R&D and the whole apparatus of people and institutions that scream “pipedream.” And instead of spending time building things, they’ve convinced people interested in these ideas that the first thing we need to do is write standards. (To engineers, this is absurd from the start—standards are things you write after you’ve got something working, not before!)

And so the “Semantic Web Activity” at the Worldwide Web Consortium (W3C) has spent its time writing standard upon standard: the Extensible Markup Language (XML), the Resource Description Framework (RDF), the Web Ontology Language (OWL), tools for Gleaning Resource Descriptions from Dialects of Languages (GRDDL), the Simple Protocol And RDF Query Language (SPARQL) (as created by the RDF Data Access Working Group (DAWG)).

Few have received any widespread use and those that have (XML) are uniformly scourges on the planet, offenses against hardworking programmers that have pushed out sensible formats (like JSON) in favor of overly-complicated hairballs with no basis in reality (I’m not done yet!—more on this in chapter 5).

Instead of getting existing systems to talk to each other and writing up the best practices,these self-appointed guarantors of the Semantic Web have spent their time creating their own little universe, complete with Semantic Web databases and programming languages. But databases and programming languages, while far from perfect, are largely solved problems. People already have their favorites, which have been tested and hacked to work in all sorts of unusual environments, and folks are not particularly inclined to learn a new one, especially for no good reason. It’s hard enough getting people to share data as it is, harder to get them to share it in a particular format, and completely impossible to get them to store it and manage it in a completely new system.

And yet this is what Semantic Webheads are spending their time on. It’s as if to get people to use the Web, they started writing a new operating system that had the Web built-in right at the core. Sure, we might end up there someday, but insisting that people do that from the start would have doomed the Web to obscurity from the beginning.

All of which has led “web engineers” (as this series’ title so cutely calls them) to tune out and go back to doing real work, not wanting to waste their time with things that don’t exist and, in all likelihood, never will. And it’s led many who have been working on the Semantic Web, in the vain hope of actually building a world where software can communicate, to burnout and tune out and find more productive avenues for their attentions.

Entertaining, brutal and accurate. Swartz does more or less succumb to the dream of the Semantic Web again in his conclusion. But who knows, maybe one day someone will deliver on it.

I picked up this handsome fellow in Texas:

I found him at Uncommon Objects, a really lovely and fun antique-y shop on Congress. They had a bunch of beautiful old radio stuff, but this was the only thing I walked away with. It’s a strange little piece — just a sheet metal enclosure for a voltmeter, probably from a test bench of some sort. Those two ports on the front were originally for connecting to the voltage being measures (though the internal wires connecting them to the meter had been snipped by the time I got to it).

It took a little work, but I’ve given this guy a brain:

 

That’s a wifi-enabled Raspberry Pi which can drive the VU meter. I also added a red and an amber LED (also controllable via the Pi).  I haven’t figured out what I want to measure just yet, but I’m sure something will come to me.

I’ve been trying to get a VU meter-driven project together for a while now, but between the software, hardware and enclosure, I was feeling a little overwhelmed. Having the enclosure taken care of made this a pleasant stepping stone.  Details/thoughts/tips:

Networking

• Linux wifi is a huge pain in the ass. Finding a card with a compatible chipset is your first major task. Lady Ada has helpfully identified a pretty cheap one and sells it here. I bought a slightly cheaper Monoprice card with my BeagleBone in mind, and it works fine with the Pi’s default Raspbian distro — no messing with firmware downloads or ‘nothin.
• Wifi cards suck up a lot of juice, and the Pi can’t supply very much over its USB ports. You might need a powered hub. The revision 2 Pi can power this adapter okay, but only if you boot with it in — plug it into a running system and the current draw will force a system reset.
• I hate messing with wpa_supplicant. The wicd package and its wicd-curses interface are your friends. But do not try to use them over SSH. The system flushes its routing when it sets up a new connection, and your SSH connection will get killed halfway through any connection to a wifi network, even if your SSH session started over ethernet. And then weird stuff will happen, like the wifi taking over the ethernet IP. It took me forever to figure out what was going on — I thought maybe the current draw of connecting to a network was bringing the system down. The moral turns out to be: use a keyboard and monitor until you’ve got a system that reliably boots and connects to the wireless network.
• I’ve got a simple Raspbian bootstrap script that updates things and sets up a Python environment. You might find it useful — though you’ll probably want to remove the part that gives my public key access to your system :-/

Hardware

• The meter was designed to measure a voltage range of 0-15V; delightfully, it still did this pretty well, despite being an antique! But the Raspberry Pi’s PWM only outputs a max of 3.3V. I had already bought a cheapo buck converter off of ebay before Timball reminded me that VU meters are basically just galvos, and use resistors to set their range. Here’s what the inside looks like:

  

  The resistor’s in the upper left. I dug out my old standby, reminded myself what a resistor network is, and added a trim potentiometer in parallel with the existing resistor (it’s the blue square on the far right of the second picture in this post). Through some careful tuning with a screwdriver I was able to make the arm sweep all the way to the 15 with just the weensy 3.3V of the maxed-out Pi. Hurrah!

• I also added a couple of status-indicator LEDs to the bottom of the meter. Their placement isn’t ideal, but you can see something when they’re on, at least. The meter wasn’t designed to accommodate them, so I had limited options.
• The LED holes and the entry point for the system’s power cable both required some Dremeling. Having sparks of hot metal showering your arm can be counted on to prompt some justified self-congratulation about wearing eye protection. Yeesh.

Software

• The wiringpi library and its Python bindings are designed to be pleasantly Arduino-like. Setup is  as easy as `sudo apt-get install -y python-dev && pip install wiringpi`. By default, you have to run your GPIO/PWM-manipulating scripts as root — there’s an alternate method available, but I couldn’t get the /sys/ GPIO interfaces to work properly in userland (though I didn’t try very hard). Ah well.
• Pay attention to the variability in pin names. wiringpi uses the pin numbers of the underlying microcontroller, not the numbers from the Pi connector. This table will help you translate between the two. Unless you’re doing something crazy, the Pi’s sole PWM comes out of GPIO1, which is “pin 1″ to wiringpi and pin 18 on the Raspberry Pi connector.
• The software I wrote to run this thing is nothing special, though there is one convenience library you might enjoy: pwmcalibrate.py. Using a curses interface, it slowly steps down the PWM output from max to zero, recording the active PWM setting as you hit spacebar, which you should do as the needle crosses each step on the meter. This data is used to create a calibration file that can be easily reloaded, allowing you to set the meter by the desired display value without having to think about it much. It also does some simple linear interpolation if you ask for a value that’s between recorded calibration steps. I’m pretty pleased with how it performs, though the meter itself seems to display some performance variability at the low end of its range. Probably nothing to be done about this, given the age of the hardware. Oh, and it will slowly step down the PWM when making transitions, if you ask it to. This is worth doing if you don’t want the needle to bounce around a ton.

I think that’s it. Now I just need to hook it up to something worth measuring. Or Twitter, I guess.

I haven’t posted my last two book reviews. In the case of Rabbit, Run, I was skittish about offering any thoughts on a classic that has already been analyzed to death by scholars smarter than me. In the case of Homeland, I wasn’t eager to express equivocal feelings about the work of an author who is professionally relevant to me. But those links work if you’re curious. Back to new stuff:

I wish I had liked this more. I love Swamplandia — Russell’s gift for language is undeniable, and when given breathing room, her characterizations amaze. The time Ava spends with the Bird Man remains among the most perfectly balanced and perfectly frightening pieces of text that I can recall.

I haven’t read her first short story collection, but I understand it to have been well-received. That leaves me puzzled by this one: it really seemed like Russell is still figuring out the form.

If you measure a story’s value by how much time the reader spends thinking about it, the first and titular story is the best. But it’s kind of a lousy piece of fiction, completely crushed by the allegory Russell wants to construct about monogamy and the inescapability of dissatisfaction. You find your Magreb, and you find some solace, and maybe it’s still not enough. That thought will stick with a guy, I admit. But Russell abandons her charms to achieve this end.

Reeling for the Empire is a fairly pat sci-fi short story, slightly sad but boring in its construction.

The Seagull Army Descends on Strong Beach is the collection’s fullest exercise in characterization, and it’s lovely in its own way. The relationships were a little too reminiscent of Junot Diaz for me to feel like Russell owned her own story, though, and the supernatural component felt vestigial.

Proving Up is scary but, again, recalled similar work done in a superior manner (McCarthy, in this case). It also suffered from plot holes and a horror movie coda that frightens but also injects regrettable narrative certainty.

The Barn at the End of Our Term and Dougbert Shackleton’s Rules for for Antarctic Tailgating offer the most amusement. Russell is extremely funny, but this is the only span where she indulges that talent. The latter story is really just a bag of jokes, an extended funny riff from a boozy evening translated to a writer’s circle. But The Barn is very fine work, I think.

The New Veterans is a mess.

And then, finally, there is The Graveless Doll of Eric Mutis. It’s about guilt: how one sinks into it like a pit of mud — with, sure, the occasional impotent struggle — but also with an outcome that’s been known and inescapable for years. Russell does this one right, but I’m not sure she can offer anything new to those who understand how this works.

So: not bad, but not great. I will reread and recommend The Barn at the End of Our Term next President’s Day. And I’ll wait and hope that the next thing Karen Russell writes once again outpaces my emotional imagination.