tecznotes

Since it's Where 2.0 and I'm not there, I'm vicariously taking part in the fun by showing how to prepare and publish paper maps for the web so they can be used in combination with some of the better-known street mapping services on the web, like Microsoft Virtual Earth or Google Maps. There are a few steps involved, including a really tedious stretch in the middle where you cross-reference points on your scanned map with known geographical locations so you can rubbersheet it into shape.

I've been doing a bunch of this recently to help my girlfriend Gem with a project for one of her sustainable urban design courses, and so far we've got an OpenLayers-based slippy map of Oakland featuring overlays from 1877, 1912, and the 1950's:

There's a bit here that's similar to the Modest Maps AC Transit tutorial, but the idea with these maps is to match them to the same mercator projection quadtree tiling scheme used by all the popular online mapping services.

Step one is to get a map. We've been finding our historical maps at the Online Archive of California, but the particular 1950's road map we added recently came from a flea market for $7. Any halfway decent flatbed scanner should get you a workable image. I scanned this one at about 600dpi in several pieces, and used Photoshop to stitch them together. I ended up with two 500MB+ TIFF images, one for pages 12-13 of the road map showing the bay shore of Oakland, the other for pages 14-15 showing the hilly bits.

Step two is the tedious part. You have to provide geographical context for the rubbersheeting step to know how your map is positioned in the world, taking into account buckled paper, surveying mistakes, and errors in scanning. For a selection of points (a dozen or more), note the geographical location in latitude & longitude and the map position in pixels, using a tool like the Google Maps Lat, Lon Popup and Photoshop's info palette. This is my coverage of the two portions of the road map:

Each of those locations is noted along with its pixel position on the map image, e.g. the pixel at (x=184, y=202) corresponds to (37.831175 N, 122.285836 W).

The program I use to do the actual geographic work is a set of open source utilities called GDAL. I started with gdal_translate, and used it to note the positions of all the points above:

gdal_translate -a_srs "+proj=latlong +ellps=WGS84 +datum=WGS84 +no_defs" -gcp 184 202 -122.285836 37.831175 -gcp 1668 50 -122.267940 37.816158 -of VRT pages-12-13.tif pages-12-13.vrt

The parts that go "-gcp (x) (y) (lon) (lat)" are repeated for each of your reference points; I had 17 for one of the images.

So that results in a VRT file, a chunk of XML that describes the geographic orientation of the image, without actually touching the image itself.

Step three is quick, just a matter of using gdalwarp to perform the actual warping and bending of the image to its new shape:

gdalwarp -t_srs "+proj=latlong +ellps=WGS84 +datum=WGS84 +no_defs" -dstalpha pages-12-13.vrt pages-12-13.latlon.tif

Now you have a new TIFF file in a known projection suitable for slicing up into the 256x256 pixel square tiles used by Yahoo! and Google and OpenStreetMap and Microsoft and everybody else doing maps online. In my case, I did the steps above for two separate images.

Step four is where a custom-written Python script swoops in a slices the map up into a folder full of tiny images. There's a bit of opaque magic here, but you'll need to get a copy of PIL, and you'll also need to tell the script where to find the GDAL programs gdalinfo, gdalwarp, gdal_translate, and the PROJ utility program cs2cs. All of these programs are available via Debian's package management system, apt. Make a directory called "out" for all the tiles, and run the script like this:

python decompose.py pages-12-13.latlon.tif pages-14-15.latlon.tif

Now wait. This part can take forever. It took a solid few hours on the virtual server where I do most of this stuff. After it was done, I posted the whole collection to a server, like this:

Once you have a pile of tiles sitting on the web someplace, get a copy of OpenLayers and set up an HTML page where your map will live. OpenLayers is one of those architecture astronaut libraries that's so full-featured, so extensive, that it's almost impossible to figure out how to do the one obvious thing you want. A bit of conversation with the developers showed that the "right" way to make a tiled slippy map in the Google projection is to pass the following arguments to the OpenLayers.Map constructor:

{ maxExtent: new OpenLayers.Bounds(-20037508.3427892, -20037508.3427892, 20037508.3427892, 20037508.3427892), numZoomLevels: 18, maxResolution: 156543.0339, units: 'm', displayProjection: new OpenLayers.Projection('EPSG:4326'), projection: 'EPSG:900913' }

There's some other futzing around in Javascript you have to do, but ultimately you end up with a map and several layers based on the OpenLayers.Layer.TMS class, like the one I'm using. I've included a layer of Microsoft tiles there for a present-day comparison, and with just three points of historical reference, a bunch of interesting patterns emerge:

• The 1877 layer shows future plans for the dredging and widening of Oakland Harbor, before Alameda was an island and before the creation of Government a.k.a. Coast Guard Island.
• The parcel grids on the 1877 and 1912 maps extend right out into the bay. They knew what they were on about, these manifest destinarians.
• The 1912 map is principally about rail transport, and there's a ton of it. Rails everywhere that today aren't much more than uncomfortably-wide streets.
• The 1950's map was published by Standard Oil, and it makes almost no mention whatsoever of the rail travel options available in Oakland. This is especially ironic, given that SO was one of three companies (GM and Firestone were the others) convicted in 1949 of criminal conspiracy to destroy rail systems like the Key Route then operating in the East Bay. In the 1960's most of the rail routes were eventually dismantled, and we're just now starting to figure out how to make up for the loss.
• The 1950's map also has no freeways, and clicking back and forth between it and the present-day layer shows exactly which neighborhoods were cut through to introduce the 80, 880, 980, and 580 highways.

On the advice of Tod Kurt, Ben, and others, I bought some shift registers to try a second pass at the Atkinson-dithered 8x8 screen. Success!

The wiring now requires just three data pins instead of the previous 12 data pins, and results in a completely addressable screen:

There are two chips on that board: one controls the rows and one the columns. If I added a third, and a mess of more wires, I could use the full red/green capacity of the LED matrix. Sadly, it's getting tangled. I think if I learned to solder it'd be possible to get all three shift registers squeezed under the matrix for a nice little screen module:

Anyone out there willing to spend an evening showing how to properly connect components to a prototype board? I can't get back to this stuff for a little while anyway, I think I may have fried the controller on the Arduino somehow.

This is the second half of a talk that Tom Carden and I did together at U.C. Berkeley's iSchool, a few weeks ago on March 20. Tom has the first half with slides posted on his own blog. He talks about the general studio/Stamen context for our forays into visual depictions of urban data, while I go deep on Oakland Crimespotting in particular. I returned to Berkeley's Graduate School of Journalism a few weeks later to deliver a slightly-edited version of this talk alone. A video that I'm afraid to watch is on Youtube here and here.

CrimeWatch

First, it's important for us to back up a bit and understand the need that Crimespotting tries to address.

Our initial work was based on the City of Oakland's existing CrimeWatch application, a "wizard-like" form-based website available at http://gismaps.oaklandnet.com/crimewatch/. The first thing a visitor to CrimeWatch sees is a disclaimer form and a few pages of instructions.

Once past the first page, there is a long series of questions that a visitor must answer: what kind of crime to view, where to look, how far back in time to search. The crime types here seem to correspond to broadly-used categories, but there are differences among jurisdictions. For example, Oakland does not publish domestic violence as a separate category. San Francisco, which uses a similar crime mapping application, has removed homicides entirely from its online maps.

Eventually, the resulting map loads in the browser. The presentation is not entirely satisfying, with a range of tiny report icons overlaid on a map and frequently obscured by information detail windows.

Parsing Images

This is the technically difficult part. The second of the two images above shows our original data input: a compressed image covered with small, graphic icons.

We used Python Imaging Library and a collection of scraping tools to search for instances of known icons and convert them to rough GPS positions.

Because we knew what we were looking for (e.g. Vehicle Theft, Prostitution, Narcotics, Robbery, Vandalism shown here) it was possible to search by shape and color to come up with positions for these icons on CrimeWatch maps.

Early this year, Oakland City IT opened up a reliable, text-based feed of crime information for us. This was a huge help, and we no longer need to go through the slow image-scraping process detailed above. Just before IT made a better source of data available to us, we were putting the finishing touches on a Mozilla browser plug-in for distributed scraping, which introduced an element of human sanity-checking after each sweep.

Permanent Links

All of the data we collect is available in a non-mapped, text form. Everything is shown on a map, but it's also possible to slice and dice the data to show reports of a certain type or from a certain date, listed out.

The single most important improvement we think Crimespotting introduced to Oakland is the concept of a permanent link for every reported crime. These permalinks contain isolated maps of the event, connected reports, a place for people to leave comments, and other reports from the same area and time.

Patterns

Street crime in Oakland really only happens between the 880 and 13 freeways. South and west of 880 is industrial, while north and east of 13 is hilly, suburban, and quite affluent. Property crimes (marked in green) happen all over, but violent crime (marked in red) is further constrained to the poorer parts of town below 580.

We've noticed that prostitution arrests seem to come in waves - long stretches of no activity, followed by rapid bursts along San Pablo Boulevard in West Oakland or International Boulevard in East Oakland. Above we can see the result of a string of busts along International in the Fruitvale area shown in blue, the color we use for quality of life crimes like drugs, alcohol, or disturbing the peace.

Although West Oakland is commonly thought to be a dangerous place, the data we've seen shows that violent crime is really spread evenly between West Oakland and downtown. There's no cluster of violence to the West, but there is a sharp cluster of drug activity (in blue) between 580 and San Pablo.

One important lesson we learned from our users: police beats are more important than we thought. This is a map of reports from beat 04X. The beat number system is how citizens interact with the police department, and what we assumed was an arcane administrative detail is really a living, breathing idea. Not so with "police service areas" or city council districts.

Outcomes

Really, the most important thing we've learned is that motivation can come from everywhere. I initially started this project because I had hurt my back in late 2006, and had a bunch of free time on my hands over that winter. What started as a technical curiosity became a studio project and eventually a public website. Motivation comes from all over, give people data and they'll figure out something interesting to do with it.

Just showing crime is relentlessly negative, and seems to really draw out the kind of graffiti-squad neighborhood busybodies who focus solely on little problems. A near-universal reaction from non-residents to this particular project has been relief that they don't live in Oakland, but it's really not that bad here. It just looks bad when all you show is crime. We'd like to map other things: city services (police, fire, emergency), tax parcels, effects of policy, other administrative information that's hugely important.

We're also thinking about how to fill in the tapestry around Oakland - currently we're covering a very small, narrowly-defined area.

On the bright side, we're using our projects to test the assumption that there's something interesting going on with city data. Decreasing costs (money, complexity) of data analysis techniques (e.g. Google's new visualization kit) drive demand for available data.

We think in the very near future, it will become cheaper for cities to publish raw data and let citizens do their own analysis with IT in a support/enabling/superhero role. Our argument is not about "democratizing" anything and the political baggage that rides along with such terminology, it's about responding to changing costs.

Matt Webb sent this to me a while back, it's the Cliff Martinez soundtrack to the George Clooney version of Stanislaw Lem's Solaris. The soundtrack is instrumental, spooky, and calming.

1. Is This What Everybody Wants
2. First Sleep
3. Can I Sit Next To You
4. Will She Come Back
5. Death Shall Have No Dominion
6. Maybe You're My Puppet
7. Don't Blow It
8. High Energy Proton Accelerator
9. Wear Your Seat Belt
10. Wormhole
11. We Don't Have To Think Like That Anymore

Also apropos of nothing, the Alan Parker edition of The Orb's Little Fluffy Clouds.

This is so dumb.

Schweddy Eagle is an application I just wrote that yanks your location from Fire Eagle and tells you what NPR stations are in your area. You could use it with a jesus phone, perhaps in a rental car, to find quality radio wherever you happen to be.

It uses the excellent OAuth protocol to safely ask for your current location.

Last October I published a tiny implementation of (Bill) Atkinson dithering in Python. Aaron ran with it and created a Modest Maps filter to prepare maps for print. Now, I'm finding Atkinson dithering useful when pumping pixels through this LED matrix I bought from Sparkfun and connected to my Arduino.

This is me messing with hardware.

First off, the matrix isn't an easy thing to wire up. The datasheet provides these two diagrams:

A bunch of trial-and-error showed that the pins on the matrix are ordered from #1 in the lower left, to #12 in the lower right, up to #13 in the upper right and #24 in the upper left: counter-clockwise, one pin for each row and two pins (green, red) for each column. The pins are grouped in sets of (green, red, row) across the top and bottom, with the top pins controlling the left-hand columns and upper rows, and the bottom pins controlling the right-hand columns and bottom rows. It's necessary to know which end is up, shown by the (now rubbed-off) product code printed on the bottom of the matrix component. I think a little extra effort on Foryard Optoelectronics' part would have resulted in a matrix that was radially symmetrical, and worked identically regardless of how it was pushed into the breadboard.

You connect a column and a row from power to ground, and the corresponding LED lights up. You can address one entire row or one entire column at a time, but a full 8x8 image requires you to scan from one to the next, illuminating pixels in sets of 8 hundreds of times per second.

I don't have 16 digital outputs on the Arduino, so I can only address six rows and six columns. This is enough to beging to experiment with tiny images, like the fades from all-off to all-on and back shown here:

There are two pieces of code making that happen. The first, matrix.pde, is running on the Arduino, set up to accept incoming bytes and display them on the little screen. It's super dumb, but it's got scan lines and an off-screen buffer to reduce flicker, so I'm happy with how it works. The second, fade.py, is running on my laptop and pumping strings of binary data over a serial connection to make the pretty pictures. At the moment, it's fading from white to black and back and sending dithered versions of those images over the wire. If my math is right, it should be able to do this at least 100 times per second without breaking a sweat, so I'm thrilled with the time resolution.

I'm not yet sure where this goes next, but I'm going to try running some simple video over the wire to see whether it's even remotely recognizeable as an image.

Two hours now, since I spoke with Twitter's Alex Payne on API design and development to a weirdly wide room at Moscone West. I think we were well received ... some excellent questions toward the end of our two-part talk. I'm exploring the idea of RESTfulness as a physical metaphor for the movement of objects to help better explain why it's winning the headspace competition against SOAP and XML-RPC.

We put our slides and notes up on Slideshare. The notes may even represent some of what came out of my mouth while I dropped into auto-pilot up there...

I've been an active user of Ffffound! since late summer of 2007 (sorry, I don't have any invites), here is a collection of some of my favorite images from the site. I've roughly categorized them into a few groups: maps, snow, geometry, mist, texture, grain, frozen moments, WTF?, and everything else.

Maps

Maps are what we do, here are some great ones.

Aaron Cope has been thinking about the papernet.

This is a map drawn on a glove that Gem found.

I love freeway interchanges viewed from above.

Snow

I have a soft spot for photographs taken through heavy snow, with flakes out of focus and huge in the foreground. Combined with the short-range lighting in a few of these, it makes for a lovely effect.

Geometry

Ambient occlusion is an interesting trend in rendering, like additive blending.

Mist

Many of my favorite images from Fffound! are photographed through mist.

Texture

These are some of Fred Scharmen's lovely branching sketches.

Grain

Visible grain is making a re-appearance in photography and film, see e.g. No Country For Old Men.

This one's from a collection of vintage color photos from the 1940's and later.

This one's in Wroclaw, my hometown.

Frozen Moments

WTF

A lot of completely absurd shit makes its way onto Fffound!.

The bear was my AIM buddy icon for a while.

This strongly reminds me of Madame Chao's noisecore visual style.

A lego rendering of Stephen Hawking!

Etc.

The girl's face on the right absolutely makes this photo.

Recently, one of my most-linked blog posts.

From the Helvetica movie.

I first found this amazing image in Netochka Nezvanova's ("the most feared woman on the internet") images directory, back when I gave two shits about the software abomination that was nato.0+55.

This lovely photo convinced me that black spokes with light-colored rims would be a great idea for my IRO.

From Generator:

The stark videos of Brandon Morse present the viewer with excercises in tension, set tableaux in which structures morph and twist under physical constraints. Stripped-down architectural forms that ought to exhibit the rigidity of highrise buildings instead engage in a tug-of-war, the result of a string simulation distributing kinetic force through a network of nodes.

Jan Chipchase, on "Sente":

Sente is the informal practices of sending and receiving money that leverages public phone kiosks and trusted networks. In Uganda the word Sente has two meanings the first being 'money' and the second 'the sending of money as airtime'. It works like this:

Joe lives in Kampala and wants to send his sister Vicky 10,000 Ugandan Shillings - about 4 Euros. He buys a pre-paid top up card for that amount but instead of topping up his own phone calls the local phone kiosk operator in Vicky's village. The phone kiosk operator uses the credit to top up his own phone, takes a commission of anywhere between 10 and 30% and passes the rest onto Vicky in cash. The kiosk operator then resells the airtime at a profit (it is after all his business).

I love the idea that this currency has a value ceiling, an upper bound defined by the number of people with minutes in the day to talk to one another.

Jack Rusher, on money:

The early Mesopotamians used a weight of barley as their first currency. It seems important to point out to the modern reader, accustomed as she must be to the way modern currency works, that this money was different from the money of today in some very important ways. It was an actual edible commodity that could be used to make soup, bread and beer, for one thing, and for another, it was prone to decay: pests ate it, it tended to rancidity if kept for too long, and so on.

This latter property of the currency was shared with most goods in the economy, all of which fell somewhere along a continuum of impermanence. The impermanent nature of these goods is linked to the underlying ecosystem from which all value ultimately arises; everything that wasn't made of sand (pottery) or metal (tools and jewelry) was the direct product of sunlight and bio-mass, and consequently subject to unavoidable near-term wear and decay.

...

In a very real sense modern economics is still suffering the effects of a 5,000 year old swindle. The modern wisdom that a small rate of inflation is part of a healthy economy comes down to the need to make our silver behave a little more like barley.

There's an upper bound here, as well, defined by how much actual barley can be consumed.

Adam, on twitter:

If I were to design a universal currency that didn't float too badly, I'd base it on avg'd cost of balanced 2000 kCal diet in 200+ markets.

How hard could it be to design a simple currency with these characteristics and set it loose? Second Life has one; seems like all that's needed is an acceptable amount of trust and an exchange rate. Great conversation with Adam exploring this earlier today.

Instead of a gold standard, it would operate on a food bank standard where you traded your tokens in for a single balanced meal when you weren't trading them for goods and services. This also reminds me of petrodollars, another concept connected to the ultimate foundation of value: fueling biological and other activity for the purposes of movement and procreation.

Here's the scene at the 7th street post office when I dropped off my taxes last night at 10pm:

Andy has a collection of fanboy supercuts, a "genre of video meme, where some obsessive-compulsive superfan collects every phrase/action/cliche from an episode (or entire series) of their favorite show/film/game into a single massive video montage." His collection includes some of the excellent and bizarre Lovelines isolation studies by Chuck Jones.

I'm reminded of how these constitute a kind of search index, a concept first introduced to me 11 years ago via Brian Slesinsky's Webmonkey article, Roll Your Own Search Engine. That was the first of many demystifications of big, web-scale technology for me. The thread running through all these fan cuts is the inverted index, identical to the concept introduced in that ancient article. An inverted index maps elements such as words to their source locations in a data corpus. Each of the pieces Andy links to is a kind of inverted index, pointing to locations of obscenities, audible inhalations, wilhelm screams, and so on.

The other thing it reminded me of was Simon Winchester's excellent book, The Professor And The Madman, an account of W.C. Minor's assistance in constructing the first edition of the Oxford English Dictionary. Minor was a confined lunatic with an extensive personal library, and the OED required that every sense of a word in its definition be traceable to an original, printed quotation. These were crowd-sourced from literate Englishmen of the time, but Minor's contribution went above and beyond because he noted interesting words as he read, constructing an inverted index of his library for OED-worthy terms. When dictionary editor James Murray needed a quotation for a particular word, there was a good chance Minor had already encountered and indexed it.

The works pointed to by Andy's blog post (and additions in the comments) are a special form of indexing, made possible by cheap communication and digital media. Let's hope the RIAA/MPAA don't fuck everything for an emerging form of media consumption.

Google launched their app engine today, and I'd like to be the 10,000,000th pundit to comment on it. First, it's clearly a nod to Amazon's thing, a direct competitor for developers looking to host projects on proper servers without incurring proper hardware costs.

Everyone else is already describing what this is, but I'm interested in the ethical and motivational implications. Most of what I'm reading about the GAE is some variation of "joy, now I have to learn Python", which I think is an accurate stand-in for Google's entire stance on this project. A quick initial read of their documentation suggests that there's a lot more than "learn Python" here - there's also "learn Django" and "learn BigTable". GAE is as much an architectural, moral, and stylistic project as it is a technical one. Where Amazon gives you shiny rack of tools to play with, Google gives you the Tao.

At the moment, Google seems to have tuned their project towards the world of web applications, not the kind of general purpose computing offered by Amazon. I expect this to change. AWS is pushing a menu of services like SQS that provide specific pieces of a distributed infrastructure. GAE is giving you the whole shooting match in one go, but telling you approximately how it should be used. I've heard a bunch of conjectures on why this is: some people think it's a way to smooth the entry path for startups looking to get bought by Google ("hey, we already use all your stuff"), while Tom sneakily suggests it's a golden parachute for soon-to-be vested ex-employees who'd still like a bit of the old infrastructure to play with.

My own initial take on both projects has been like night and day. Amazon's services were like a breath of fresh air while so far, Google's has filled me a dread I dare not name, in spite of proudly using Python as my "thinking language" of choice. AWS exists happily as a component set for other applications, and I use S3 extensively to serve map tiles and listen to music while Crimespotting runs on EC2. I think that in this case, Google is commoditizing the wrong end of the stack. They seem to be providing the equivalent of single-language shared hosting without really opening up the benefits of a massive computing infrastructure that a tiny minority of applications need or want. I take my own initial preemptive exhaustion as a sign that they are expecting too much of their prospective users. Kiss the ring.

That said, both of these services have an ethical dimension that I appreciate. I trust that machine instances and running applications not seeing a lot of activity are swapped out in favor of those that are, a form of carbon footprint minimization impossible to achieve with your billed-monthly colocated server. In this case, scale does matter as long as the two companies keep their prying eyes out of the data and processes entrusted to them. I'm looking forward to seeing greater commoditization in this area, and I happen to think that Amazon is doing a significantly better job moving us in that direction.

Muxtape beat me to the New Amazon S3 POST Support + MP3 land rush, but here's mine anyway. It's currently constrained by pokey hotel internet, and will be filling up over the coming hours.

Update: I don't think they're actually using S3 POST at all, which means they're probably slamming their server for no good reason.

Update #2: A few example muxtapes from people I know: tma (Tomas), mattb (Matt), plasticbaguk (Tom), blackbeltjones (another more different Matt), girlwonder (Molly).

Update #3: Fastest spreading intermeme pretty much ever, I bet muxtape.com wishes they had used S3 POST: bopuc (Boris), neb (Ben), listentomy (Aaron), compulse (Shawn).

Update #4: "memetape": revdancatt (Dan), straup (Aaron).

We did our UC Berkeley Visual Urban Data talk on Thursday, and from this side of the podium, it went really well. Tom covered the first half with a reverse-chronological overview of Stamen's mapping work. We chose to go backwards to see whether it was a more effective presentation when the most up-to-date work was presented first, with the remainder acting as background. I then went in-depth about Oakland Crimespotting, describing the project's brief history and a number of lessons we've learned from it.

Turnout was excellent, and my first time doing a one-topic talk was loads of fun. This is definitely something I want to do again: taking a full hour to deeply explain a particular slice through our work is a welcome change from our usual rapid overview presentations. Hopefully video will be available soon, and we'll have a chance to revise and present the material a second time in another context.

The reverse-chronological order was partially inspired by Norman Davies' Heart Of Europe:

Heart of Europe: A Short History Of Poland makes no pretense of presenting a full and balanced survey of Polish affairs over the last thousand years. Although each chapter contains a brief chronological narrative, the emphasis has been firmly placed on those elements of Poland's Past which have had the greatest impact on present attitudes. ... For similar reasons, the main chapters have been written in reverse chronological order. ... In this way, the narrative leads from the more familiar to the less familiar.

Tom and I are speaking at UC Berkeley's School of Information tomorrow:

Visual Urban Data: A Journey Through Oakland Crimespotting

• 110 South Hall, Berkeley CA
• Thursday, March 20, 2008
• 5:15 PM - 6:30 PM

We'll be talking mostly about Oakland Crimespotting, with some diversions into the rest of Stamen's mapping and urban work. Come join us!

I connected one of these to one of these, and made it do this:

Hardware is kind of a bitch. I have all these resistors that I'm told are necessary to prevent things from asploding, and if I want to have more than two rows of LED on there, I'll have to purchase a few more transistors. But, so far, encouraging!

Bruno Latour's investigation of the stillborn French transit system Aramis was recommended to me by Mike Frumin after another recent book post. Latour dissects the abandonment of Aramis in the form of an academic mystery novel, featuring a cranky sociology researcher/investigator and his young, upstart graduate intern ploughing through two collected decades of first-hand accounts from technologists and bureaucrats. The Aramis PRT occasionally speaks up for itself in the voice of a temporally transposed Frankenstein's Monster.

Latour builds on his translation model from Science In Action and shows how a successful technology project changes in response to its environment, taking on new features and satisfying new needs as it navigates the landscape of human and non-human actors from conception to delivery. No change, no project. Stasis.

Aramis in particular is an example of a late-1960's fad in public transportation that sought to marry the convenience and flexibility of the automobile to the high volumes and socialized costs of mass transit. PRT's never really panned out despite multiple attempts, though in some ways the current crop of car-sharing services is fulfilling this dream from the opposite direction. In this particular case, Latour shows how Aramis was never moved past the "technically sweet" stage: always an engineer's dream, worked on by idealists with little interest in taking its revolutionary technological concepts and adapting them to the physical, financial, and political realities of Paris in the 1970's and 80's.

I marked a lot of pages here, and this is a long post. Normally I'd try to summarize everything but Latour is a lucid, enjoyable writer and his prose is a joy, so this small bit of background should be enough.

Page 24, on state:

The observer of technologies has to be very careful not to differentiate too hastily between signs and things, between projects and objects, between fiction and reality, between a novel about feelings and what is inscribed in the nature of things. In fact, the engineers the observer is studying pass progressively from one of these sets to another. The R-312 was a text; now it's a thing. Once a carcass, it will eventually revert to the carcass state. Aramis was a text; it came close to becoming, nearly became, it might have become, an object, an institution, a means of transportation in Paris. In the archives, it turns back into a text, a technological fiction.

Page 45, on variability, solidity, and resilience:

Mr. Legardere may vary in size, the ministry will change hands ten times - it would be unwise to count on stability there; but the signatures and stamps remain, offering the alliances a relative durability. Scripta manent. That will never be enough, for signed documents can turn back into scraps of paper. Yet if, at the same time, the interlocking of interests is actively maintained, the law offers, as it were, a recall effect. After it is signed, a project becomes weightier, like a little sailboat whose hull has been ballasted with some heavy metal. It can still be overturned, but one would have to work a little harder to prevent it from righting itself, from returning to its former position. In the area of technologies, you cannot ask for more.

Page 59, on metaphor:

"It's a confusion of genres," I said, forgetting my place. "Chips don't talk any more than Chanticleer's hens do. People make them talk - we do, we're the real engineers. They're just puppets. Just ordinary things in our hands." "Then you've never talked to a puppeteers. Here, read this and you'll see that I'm not the one getting carried away with metaphors. Anyway, do you know what 'metaphor' means? Transportation. Moving. The word metaphoros, my friend, is written on all the moving vans in Greece."

Page 67, on the moveable frontier:

The frontier between "the bulk of the work" and "fine-tuning the details" remains in flux for a long time; its position is the object of intense negotiation. To simplify its task, every group tends to think that its own role is most important, and that the next group in the chain just needs to concern itself with the technical details, or to apply the principles that the first group has defined. Moreover, this way of looking at things is integrated into project management.

Page 72, defining "innovation":

Here is the difference between a project that is not very innovative and one that is highly innovative. A project is called innovative if the number of actors that have to be taken into account is not a given from the outset. If that number is known in advance, in contrast, the project can follow quite orderly, hierarchical phases; it can go from office to office, and every office will add the concerns of the actors for which it is responsible. As you proceed along the corridor, the size or degree of reality grows by regular increments. Research projects, on the other hand, do not have such an elegant order: the crowds that were thought to be behind the project disappear without a word; or, conversely, unexpected allies turn up and demand to be taken into account.

Page 88, time is what is counted:

The time frame for innovations depends on the geometry of the actors, not on the calendar. ... Is VAL's time the same as Aramis'? No, even though 1975, 1976, 1977, 1979, and 1980 are critical years for both. It's no good taking out a chronometer or a diary so you can measure the passage of time and blame the first project for going too quickly and the second one for going too slowly. The time of the first depends on local sites, on Notebart's role as engine, on Ferbeck, and on Matra, just as the time of the second depends on the absence of sites, on hesitation over components, on the motor's fits and starts. All you have to do is reconstruct the chain of permissions and refusals, alliances and losses, to understand that a project may not budge for a hundred years or that it may transform itself completely in four minutes flat. The obsession with calendar time makes historians sprinkle technologies with agricultural metaphors referring to maturation, slowness, obsolescence or germination, or else mechanical metaphors having to do with acceleration or braking. In fact, time does not count. Time is what is counted. It is not an explanatory variable; it is a dependent variable that needs to be explained.

Pages 109-110, on tinkering and engineering:

"But wait a minute," I exclaimed, indignant at so much bad faith and because, by chance, I had read Levi-Strauss for my exams. "Levi-Strauss contrasts modern engineers with mythical tinkerers. We engineers don't tinker, he says. We rethink all programs in terms of projects. We don't think like savages." "Hah!" Norbert muttered ironically. "That's because Levi-Strauss did his field work in the Amazon rain forests, not in the jungle of the Paris metro. What he says about tinkerers fits engineers to a T, his ethnologist's bias notwithstanding. ...when everything is going along swimmingly; of course, then it's as if there were 'experts' quite unlike tinkerers and negotiators. But at the end, only at the end. And since Aramis wasn't lucky enough to have such an end ... No, believe me, you don't have those who tinker on one side, and those who calculate on the other."

Page 118, Matra's M. Freque on arguing:

"The arguments sometimes got pretty lively. You heard everything: 'Greedy industrialist!' 'Profiteers!' 'Assholes!' But in the long run we reached an agreement. The problem with Aramis is that not enough people yelled at each other. Below a certain level, that's not good. You see, sometimes my ideas got rejected, other times I came out the winner; sometimes things got simplified, other times they got complicated. That proves it was a real debate, a real negotiation."

Pages 126-127, on mobilization:

As a project takes shape, there is an increase in the number, quality, and stature - always relative and changing - of the actors to be mobilized. Petit was just one highly placed official. Now ministers and presidents are involved. By moving from conceptual phases to production phases, you move from saints to the God they serve. Since the project is becoming more and more costly, since it is agitating more and more people, since it is mobilizing more and more factories, since the nonhumans it has to line up are numbered in the thousands, since it is a matter no longer of plowing up a beet field but of tearing up parts of southern Paris, actors capable of providing resources adequate to the new scale must henceforth be reckoned with. Ten times as many actors are now needed for the project, and they cannot be recruited one by one - one pipe smoker after another, one iron bar at a time. We have to move from those who represent small numbers to those who represent large numbers.

Page 157, Aramis speaks for itself:

Why reject me? Have I not been good? Was I not born well-endowed with virtues, unlike my brother VAL? Have I not been the dream, the ideal? What pains were not taken for my conception! Why recoil in horror today? Did not all the fairies hover over my cradle? Oh, my progenitors, why did you turn your heads away, why do you confess today that you did not love me, that you did not want me, that you had no intention of creating me? ... Of all the sins, unconsummated love is the most inexpiable. Burdened with my prostheses, hated, abandoned, innocent, accused, a filthy beast, a thing full of men, men full of things, I lie before you. Eloi, eloi, lama sabachthani.

Pages 159-160, on private doubt:

"To account for this survival, this delay, we have two elements: up above, in the higher spheres, everyone is now in favor of Aramis, unanimously. Although everybody has private doubts about the project, they give it their own backing, however half-heartedly, because they see all the others supporting it enthusiastically. Down below, with the technicians, everybody is skeptical..." "At least that's what they're saying now. At the time, no one noticed the skepticism..." "Exactly. Everybody was skeptical, but only in private. That's the whole problem: half-doubts are all scattered, isolated, buried in notes that we are often the first to see, in any case the first to bring together as a whole."

Page 174, on smoothness:

Let's calculate the sum of forces - using this expression to designate both the work all the actors do to sum up and the diversity of the ontological models they use. Let's add the thrusts of human labor, the fall of ballistic missiles, the responsibility of promises, amorous seduction, the shame of more killing, vanity, business - everything that makes Aramis impossible to suspend. Yes, it's definitely a strange monster, a strange physics. It's the Minotaur, plus the labyrinth, plus Ariadne and her thread, plus Daedalus, who is condemned to die in it and who dreams of escaping. They're really fun, those people who write books in which they think they're castigating technology with adjectives like smooth, cold, profitable, efficient, inhuman, irreversible, autonomous! These insults are qualities with which the engineers would be delighted indeed to endow their hybrid beings. They rarely succeed in doing so.

Page 180, on bureaucracy:

To make fun of the files of the bureaucrats, to make fun of the two-page notes of synthesis and the thousand-page appendixes, is to forget the work of stabilization necessary to the interdefinition of the actors. It is to forget that the actors, large or small, are as lost in the action as the investigator is. The human sciences do not show up as the curtain falls, in order to interpret the phenomenon. They constitute the phenomenon. And the most important human sciences, always overlooked, include accounting, management, economics, the "cameral sciences" (bureau-graphy), and statistics.

Page 199, on common sense:

"Everything happens in defiance of common sense, but there is no common sense for innovations, since they happen, they begin, they invent common sense, the right direction, the correct procedure."

Page 213, on figure-ground reversal:

Where is this thing, the microprocessor, to be situated? On the side of human beings? No, since humans have delegated, transcribed, inscribed their qualities into nonhumans. On the side of nonhumans, then? Not there either. If the object were lying among nonhumans alone, it would immediately become a bag of parts, a heap of pins, a pile of silicon, an old-fashioned object. Thus, the object, the real thing, the thing that acts, exists only provided that it holds humans and nonhumans together, continuously. ... On the one hand, it can be said to hold people together, but on the other hand it is people who hold it together.

Page 280, on stasis:

The report presented the 1987 Aramis, word for word, as identical to Petit and Bardet's 1970 Aramis. I found myself twenty-one interpretations, but the technological documents remained mute about this dispersion. Aramis had not incorporated any of the transformations of its environment. It had remained purely an object, a pure object. Remote from the social arena, remote from history; intact. This was surely it, the hidden staircase Norbert had been looking for. Its soul and its body, as he would say, never merged.

Page 292, on Aramis unloved:

"Yet in spite of its fragility, its sensitivity, how have we treated it? Like an uncomplicated development project that could unfold in successive phases from the drawing boards to a metro system that would run with 14,000 passengers an hour in the south Paris region every day; twenty-four hours a day. Here is our mistake, one we all made, the only one we made. You had a hypersensitive project, and you treated it as if you could get it through under its own steam. ... You believed in the autonomy of technology."

Page 295, Aramis speaks again:

Of what ends am I the means? Tell me! you hid from one another in order not to admit that you didn't want me. You built the CET the way human couples produce one child after another when they're about to divorce, trying to patch things up. What horrible hypocrisy, entrusting to the whimperings of the most fragile of beings the responsibility for keeping together creatures that are much stronger than itself.

I amused myself this weekend by pulling maps from the Online Archive Of California and grinding them up with Python and gdalwarp to make map tiles:

We launched Oakland Crimespotting back in August, and all was well for a short time. There were friendly mails from Pete Wevurski, John Russo, and others who liked what we were up to. Unfortunately, we ran afoul of Oakland's website availability, and by late October it became completely impossible for us to collect data at a sustainable rate. We closed up shop and replaced the front page of the site with an apology and a promise.

After several months of general stagnation, Oakland City IT reconnected us to a current, reliable, and accessible data source in January, and I can now confirm that it all Just Works.

There are a few bits of New sprinkled throughout the site.

We've added pages for individual police beats, such as this one for 04X, where I live. A large number of our users asked for these, though truthfully it wasn't something I expected. I've been historically critical of the forms-first approach that CrimeView Community takes ("Easy wizard interface"), eschewing it in favor of a maps-first approach. Changing standards of cheapness are a recent interest of mine, and it's cheaper to show everything. Expect to hear more of this from Tom at E-Tech tomorrow. In fact, Police Service Area and City Council District aren't ways that Oakland residents commonly locate themselves. The Police department is organized into beats, and this turns out to be the right way to interface with them if you're a concerned, active citizen. Each beat has a consistent set of officers and public contact information. Oakland CTO Bob Glaze told me the beat designations haven't changed in decades. Clearly, maps and data for individual beats were going to be necessary.

Each beat page features a map of recent reports in that area. These maps are the result of Aaron's heroic work in extending Modest Maps' static mapping abilities. WS-compose is now a sweet little map generator that will happily report geographic dot locations in HTTP response headers if you ask it nicely, among other tricks.

There are also per-beat news feeds and downloadable spreadsheets of detailed information for neighborhood crime prevention councils.

The other addition is a proper comment feature. In the past, we've had an error report form on each crime report page where residents could alert us to improperly-placed reports or other mistakes, but this wasn't as effective as it could have been. The primary problem was that posting an error report didn't really set off any alarm bells, and it certainly didn't appear on the site anywhere. I've grown to feel that replacing a clunky web interface with a mute one isn't necessarily much of an improvement, so it's valuable to provide a direct feedback mechanism right there on the site.

The error reports have now been replaced by actual comment forms where you can leave your name, a message, and an optional link at the bottom of each individual report page. The comments are keyed on the case number, so case numbers with multiple reports share a set of comments. Right now these just look like regular blog comments, but the intent of the link is to add news articles or connect reports to one another. I hope very much to see this feature of the site grow into something interesting and unexpected.

Here is the mail I sent last month announcing our return:

Hello Everyone,

We're happy to announce that Oakland Crimespotting is back, thanks to the generous help of Oakland's City Information Technology Department. After three months without access to report data, we've been granted a reliable, regularly-updated source of crime report information. This is great news: it means that the website is back up and running with current information, e-mail alerts and RSS feeds work again, and we at Stamen Design can explore new ways of presenting and publishing this important information.

Here are a few things you can do, now:

Visit the site at http://oakland.crimespotting.org/. View a map at http://oakland.crimespotting.org/map/. Sign up for alerts at http://oakland.crimespotting.org/alerts.

We are also interested in what additions to the site you would find useful or interesting. So far, we've had a number of suggestions that we're actively looking into: spreadsheet-friendly downloads, details on individual police beats, a search function, and more than one month's worth of data. If you have any thoughts on these or other ideas, send us a mail at info@crimespotting.org.

Our return would not have been possible without the help of a few key people. Ahsan Baig, Ken Gordon, and Bob Glaze at Oakland City IT built and published a source of information for us. Ted Shelton, Charles Waltner, and others helped us navigate the difficult waters of City Hall communications. Jason Schultz, Ryan Wong, Karla Ruiz, and Jeremy Brown at U.C. Berkeley Law School helped us understand how to best approach city governments for information. Kathleen Kirkwood and Pete Wevurski at The Oakland Tribune helped us understand the journalistic context of the project. Dan O'Neil and Adrian Holovaty at EveryBlock.com were a valuable sounding boards for ideas

Two weeks ago, I posted the faumaxion slippy map, an interactive interpretation of Buckminster Fuller's Dymaxion World Map. I was curious to see whether the continuous re-orientation of the map would be jarring or confusing to users. Based on some helpful feedback, I've updated the map so that the dragging and rotation behaviors are separate. Instead of continuously re-orienting itself to face North for whatever point happens to be in the center of the map during a click-and-drag, a tiny compass rose shows which way the map will rotate itself once the mouse is released. This version feels calmer, and makes for a more predictable (and therefore better?) interaction:

Need to get these out of my tab pile ... You And Your Research, What single book is the best introduction to your field (or specialization within your field) for laypeople?, digital elph sd1000, Social Design Notes: An Introduction to Information Design, What Specifically Do Generalists Do?, Systeme D Cartography, Movement, Web Directions North 2008 closing keynote, analysis of competing hypotheses, Arduino troubleshooting.

Until his death in 1993, David Pye was a professor of furniture design at the Royal College of Art in London. The Nature And Art Of Workmanship is a guide to his theory of workmanship as distinct from design. The tone of the book is slightly musty, frequently dipping into old-mannish complaints that ring slightly of "the kids today", but on balance Pye is a clear writer with a coherent idea to communicate.

The book focuses on laying to rest the fallacy of "things done by hand" in favor of the terminology of risk and certainty. These two terms form the core of Pye's theory of workmanship, and boil down to "can it be fucked up?" For Pye, the meaningful distinction is whether a thing is a result of a risky process, or a certain one. The former requires dexterity and judgement while the latter requires an assembly line and planning. The division was a new one to me, but it has occasionally snapped into focus since I started this book over Christmas, as when reading Jeff Veen's latest blog post on Indi Young's new book:

In the end, using Indi's process, we were able to convince teams that we weren't researching all the creativity out of their projects. We were researching the risk out. And no matter how the industry is faring, that's a story people want to hear.

For what it's worth, Stamen is teetering on the cusp of this distinction (among other cusps we teeter on) as we investigate the sense of formalizing our process with an explicit producer role. Thus far, our work has been raw risk. I don't mean to say that we routinely snatch victory from the jaws of defeat, but we consciously lack a "process" as someone like Jeff, Indi, or the company they helped found might understand it. It's personally interesting to me that computation and programming can still be seen to be risky in the same way that woodworking or pottery can, especially with the rapid growth of social websites whose success can not be measured by technical means alone.

(slide nabbed from Scaling Twitter)

As we take on larger slices of work, there's a natural inclination to manage risk by introducing certainty into the workflow. Namely, developing a process, knowing whether we're sticking to it, and starting think about hiring as filling holes rather than seeking out fellow travelers. I offer no opinions on this, except to say that it's an active debate.

It's also worth noting that Pye is no dogmatic fan of doing things the hard way. He devotes a number of pages (some excerpted below) to exploring why precise workmanship has been historically valued, notes that much work traditionally thought of as "hand labor" is really as jigged and regulated as machine work, observes that in many settings certainty and uniformity are desirable, and takes the Arts and Crafts movement itself to task for misunderstanding the potential joy inherent in competent work.

Page 17, on design, workmanship, and defining terms:

In the last twenty years there has been an enormous intensification of interest in Design. The word is everywhere. But there has been no corresponding interest in workmanship. ... This has not happened because the distinction between workmanship and design is a mere matter of terminology or pedantry. The distinction both in the mind of the designer and of the workman is clear. Design is what, for practical purposes, can be conveyed in words and by drawing: workmanship is what, for practical purposes, can not. In practice the designer hopes the workmanship will be good, but the workman decides whether it shall be good or not. On the workman's decision depends a great part of the quality of our environment.

Page 20, on risk, certainty, and defining more terms:

If I must ascribe a meaning to the word craftsmanship, I shall say as a first approximation that it means simply workmanship using any kind of technique or apparatus, in which the quality of the result is not predetermined, but depends on the judgement, dexterity, and care which the maker exercises as he works. The essential idea is that the quality of the result is continually at risk during the process of making; and so I shall this kind of workmanship, "The workmanship of risk": an uncouth phrase, but at least descriptive. ... With the workmanship of risk we may contrast the workmanship of certainty, always to be found in quantity production, and found in its pure state in full automation. In workmanship of this sort the quality of the result is exactly predetermined before a single salable thing is made.

Page 25, on doing by hand:

Things are usually made by a succession of different operations, and there are often alternative ways of carrying any one of them out. We can saw, for instance, with a hand-saw, an electrically driven band-saw, a frame-saw, and in other ways. To distinguish between the different ways of carrying out an operation by classifying them as hand- or machine-work is, as we shall see, all but meaningless. ... The source of power is completely irrelevant to the risk. The power tool may need far more care, judgement and dexterity in its use than the hand-driven one.

Pages 32-33, on roughness:

In the workmanship of risk rough work is the necessary basis of perfect work, just as the sketch is of the picture. The first sketchy marks on the canvas may become the foundation of the picture and be buried, or they may be left standing. Similarly the first approximations of the workman may afterwards disappear as the work proceeds, or they may be left standing. For the painter and the workman it is sometimes difficult to know when to stop on the road towards perfect work, and sooner may be better than later. In the workmanship of certainty, on the other hand, there is no rough work. The perfect result is achieved without preliminary approximation.

Pages 49-50, on design intent:

The intended design of any particular thing is what the designer has seen in his mind's eye: the ideally perfect and therefore unattainable embodiment of his intention. The design which can be communicated - the design on paper, in other words - obviously falls far short of expressing the designer's full intention, just as in music the score is a necessarily imperfect indication of what the composer has imaginatively heard. The designer gives to the workman the design on paper, and the workman has to interpret it. If he is good he may well produce something very near the designer's intention. If the workman is himself the designer he almost certainly will (but that does not imply that the designs a workman intends are necessarily good ones).

Pages 58-59, on the origins of precision:

In nature we see varying degrees of disparity between the idea and the achievement wherever we look. To Plato it may perhaps have seemed that things would look better if there were no such disparities. We, having lived in an age where to all appearances such disparities really can be banished from our environment, may doubt it. ... Our traditional ideas of workmanship originated along with our ideas of law in a time when people were few and the things they made were few also. For age after age the evidence of man's work showed insignificantly on the huge background of unmodified nature. There was then no thought of distinguishing between works of art and other works, for works and art were synonymous. ... Then and for a long time afterwards - and even now in some remote places - all the things in common use for everyday purposes were of fairly free or rough workmanship and anything precise and regular must have been a marvel, amazing and worshipful.

This reverence for precision had, I think, two explanations. ... The second, and I believe deeper, reason lay in the opposition of art to nature. The natural world can seem beautiful and friendly only when you are stronger than it, and no longer compelled with incessant labor to wring your livelihood out of it. If you are, you will be in awe of it and will propitiate it; but you will find great consolation in things which speak only and specifically of man and exclude nature. When you turn to them you will have the feeling a sailor has when he goes below at the end of his watch, having seen all the nature he wants for quite a while. Precision and regularity, in those days signified that, to the extent of his intellect, man stood apart from nature, and had a power of his own.

Page 62, on spatial frequency and diversity:

It is a matter of the greatest moment in the arts of design and workmanship that every formal element has a maximum and minimum effective range. In can only be "read" - perceived for what it is - by an observer stationed within those limits. ... In nature, as in all good design, the diversity in scale of the formal elements is such that at any range, in any light, some elements are on of very near the threshold of visibility: or one should say, more exactly, of indistinguishability as elements. As the observer approaches the object, new elements, previously indistinguishable, successively appear and come into play aesthetically. Equally, and inevitably, the larger elements drop out and become ineffective as you approach. But new incidents appear at every step until finally your eye gets too close to be focused. The elements that at any given range, long or short, are just at the threshold, that we can just begin to read, though indistinctly, are of great important, aesthetically. They are perhaps analogous to the overtones of notes.