tecznotes

Update: I've written a followup to this post.

-

This has been a strange year to live in Oakland. The FBI says violent crime is up, but the City doesn't publish statistics in a particularly friendly format. Inspired by Adrian Holovaty's Chicago Crime, I'm in the process of exploring how the new CrimeWatch II application can be bent into a more usable shape.

This post describes the first two steps in extracting information out of CrimeWatch II: downloading known maps, and extracting positions of crime markers from those maps.

Linked text files:

• Example HTTP transcript: oaklandnet-http-transcript-02.txt (19K)
• Map request shell script: map-download.txt (6K)
• Example icon: vandalism.png (0.3K)
• Final icon matching script: scan-image.py (10K)

Initially, I had expected this to be a simple screen-scraping project, but as it turns out the available data is published in JPEG form as a series of icons overlaid on simple city maps:

Before attempting to geolocate individual crimes, it would be necessary to pick them out of the map image. Because the CrimeWatch application uses frames, cookies, background-images, and other techniques, it was first necessary to log HTTP traffic and understand what requests and parameters generated each map.

I used the FireFox plug-in LiveHTTPHeaders to pick apart the process of downloading a map. This resulted in an HTTP log such as this one. Getting maps from CrimeWatch is a three-step process: first they require that you accept their terms of use by posting a simple form response, then you choose your search parameters (e.g. zip code, crime type, dates, etc.), and finally you pick through the response HTML for a reference to the map JPEG. I created a basic shell script to walk through these steps. It uses cURL and some basic HTML-parsing Perl to find the map images.

GET /crimewatch/map.asp?mapfunction1=51…
Host: gismaps.oaklandnet.com
User-Agent: Mozilla/5.0 (Macintosh; U; …
Accept: text/xml,application/xml,applic…
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,…
Keep-Alive: 300
Connection: keep-alive
Referer: http://gismaps.oaklandnet.com/…
Cookie: OmegaCVCDisclaimer=yes; ASPSESS…

HTTP/1.x 200 OK
Date: Sat, 23 Dec 2006 22:35:36 GMT
Server: Microsoft-IIS/6.0
Content-Length: 1763
Content-Type: text/html
Cache-Control: private

Once the map image has been retrieved, it's necessary to pick through the returned image and identify the crime icons (see the right-hand legend in the screen shot). Because JPEG is a lossy-compression format and many icons occlude one another, it would not sufficient to search for exact pixel matches with the known set of icons. Instead, a more fuzzy, "best fit" method was needed.

I used Python with a combination of PIL and Numeric to extract icons from map images. I created average representations for each type of icon (e.g. "vandalism", PSD and PNG) based on maps with many instances of each.

Visual feature extraction is a two-step problem: quickly find a set of possible locations for icons, then check each in detail to determine if it's a match. The first step needs to eliminate as much background noise as possible for the more time-consuming second step to work quickly. I tried three different approaches before settling on one that seemed to work best.

A first, I tried to simply eliminate background pixels to cut down on the number of possible icon positions. It was easy to find representative colors for land, water, freeways, and parks, and eliminate about 70% of the total map pixels. With approximately 1-megapixel maps, this left about 300K pixels to check for icon matches, a prohibitively large number.

Next, I tried the opposite approach: find representative colors from each icon, and find likely locations based on the presence of those colors, rather than the absence of background colors. This means that I didn't spend a lot of time checking blue "simple assault" parts of the image for obviously-incorrect red "aggravated assault" icons. This was a major time savings, since most icons contain a representative color that appears little on the map. The exception to this rule is "robbery" and "burglary", two crimes that use black & white icons. Searches for these icons take dramatically longer than the others.

Finally, I expanded on the icon matches to better account for partially-occluded icons. When I encounter a possible match that's not strong enough to be included as a complete icon, but is still within about 50% of the threshold, I check the top, bottom, right, and left halves of the icon individually. If any of these result in an above-threshold match, I include them in the final results.

The final Python script takes about 2 minutes to convert the first linked image below into the second. Partially-occluded icons are marked with a light outline, fully-matched icons are marked with dark:

Although there are a few misses, the input image represents an unrealistic worst-case: two weeks' of data covering all possible crimes, with zip code boundaries visible. It is simple to request single-crime maps, with no boundaries, for one-day spans to cut down on the icon overlap.

There are two obvious next steps: use known addresses and intersections to geocode the matched icons (the CrimeWatch application promises only that they are placed at block-level accuracy), and make further HTTP requests for more detail about each crime, especially the time of day at which it was reported.

Also, it appears that the SFPD has recently switched to a crime mapping application developed by the same vendors, Omega Group and MoosePoint (!), so expanding this process to cover San Francisco should be easy.

-

Keep reading the followup to this post.

I'm thrilled that the New York Times decided to publish a redacted editorial in this format:

Knowing that information is being edited here is crucially important. See also this flyer design.

Oakland, one mile west of my house:

Two men were killed and four wounded in two shootings that happened a half-hour apart Wednesday night in East and West Oakland, police said. The shootings do not appear to be related, police said Thursday. At 8 p.m. Wednesday a gunman opened fire on a car driving in the 600 block of Apgar Street in West Oakland, killing driver Lord Addo, 21, and wounding his passenger.

-Alameda Times-Star

Piedmont, one mile east of my house:

For weeks, Piedmont police were stumped by the Beanie Baby bandit. The popular stuffed animals were mysteriously showing up overnight on porches and in the yards of two homes on Rose Avenue. One of the families feared a stalker. Now, police in the tranquil East Bay city think they may have identified a suspect: Gertie, one of the family's cats, which was caught on a surveillance camera carrying the plush toys in its mouth. ... "The cat was caught on tape, but we don't know if it was moving the Beanie Babies from the location where somebody put them," police Capt. John Hunt said Thursday.

-SF Chronicle

This is a quick followup on a previous post about specifying time. I've created a minimal Actionscript 2.0 library that parses and generates time ranges in that form: timespan.tar.gz, 2.5k. Combined with Kevin Lynch's technique for stateful linking in Flash, it should be easily possible to read, use, and update links of the form example.com/#2006-12-19T21:10:00Z/1D0:01:00, for linking to absolute times and timespans in Flash files.

Monolake, Live at Frigid Sydney: part 1, part 2. 60 minutes, minimal techno, very icy.

Things currently open in my browser, that I want to read eventually, don't have the time for, and won't eject into the wasteland of Del.icio.us:

• New York Times Year In Ideas
• Indri: Language modeling meets inference networks.
• Jakob Verbeek
• Bray-Curtis Algorithm
• PennAspect PLSA
• PostDocs on InfoViz
• InfoViz Evaluation Papers
• Boot chart for Linux
• Dimitre's photos tagged with "spamart"

Same deal, in Preview:

• A Flat World, A Level Playing Field, a Small World After All, or None of the Above?, Review of Thomas L Friedman, The World is Flat, Edward E. Leamer
• Is There Science In Visualization?, associated slides
• Hofmann '99: Probablistic Latent Semantic Analysis

Ableton's Live music software has a great time navigation widget. It's a fairly standard interface for viewing slices of a sequenced composition, similar in spirit to Google Finance. They integrate zoom and pan into a single interface element, mapping vertical motion to zoom (up = zoom out, down = zoom in) and horizontal motion to pan left and right. The two don't occur together, so if you start moving horizontally you don't get to zoom until you make a sharp vertical motion. I suspect there's a cut-off at each 45 degree angle.

Anyway, here is a brief video showing how it works:

• Small, 2.6MB
• Actual size, 5.1MB

It's unfortunate that the mouse cursor has to disappear when dragging.

This old (new to me) video is out of control: Bike Messenger Race, New York City

What the hell is going on there? Seven minutes of P.O.V. stoplight-running and squeeezing between cabs and busses in the busy streets of Manhattan, that's what. Although I lack the cojones to do something like this myself (and suspect that the excitement is being pumped up a bit with a fisheye lens anyway), what a pleasure to watch. It's like the Star Wars trench run but more visceral, because you feel like one of these riders is going to eat it or get doored I.R.L. at any moment. They weave between pedestrians, shimmy out of the way of cross-traffic, and hold on to vehicles for an extra power-up.

The traffic moves are clearly deadly, but having just visited New York I can say that they're better drivers over there for being as aggressive as they are. Here in San Francisco, I get pissed off at drivers or pedestrians for being wishy-washy: go, no-go, hold on, on the phone, on the pot, etc. In New York everyone just goes, so you pretty much always know what to expect and how to behave. Then again, with asshole bikers like the guys in this video, you end up with something approaching this classic video of traffic in India.

Of course this is all academic for me at the moment, because I have a raging pinched nerve in my leg and it's making me an utter basket case. Can hardly sit or walk, and especially can't ride my bike.

I've been trying to think of ways to denote points and ranges in time suitable for use in URL's. Notation should also be human-readable, free of URL "problem" characters (e.g. spaces, ampersands), guessable/hackable, and universally equivalent (i.e. no time zones!). One of the possible contexts is for use in Flash, which is aware of only two time zones: local and UTC. It and Javascript also have clumsy and primitive string support, so anything that requires regular expressions or a state machine is out.

Tom and I kicked around a few simple possibilities today:

1. Point: 2006-12-01T06:12:01Z.
2. Range, point-to-point: 2006-12-01T06:12:01Z/2006-12-02T06:12:01Z, a machine-friendly way to do ranges.
3. Range, point-distance: 2006-12-01T06:12:01Z/1D00:00:00, a more human-readable way to do ranges like "one week".

WMS seems to do something similar, with the same RFC3339-derived stamps chosen for Atom. The distance part of the point-distance range syntax is similar to Python's timedelta string representation.

Too limited, or appropriately limited?

5 minutes ago - Kati Kim and her two girls have been found, they are being helicoptered out to a hospital. James Kim went out on foot two days ago and the Search and Rescue team are now tracking him and hope to find him soon.

This is a lazyweb request for a math tutor. I know just enough linear algebra and matrix math to be curious, but not yet enough to be dangerous, and I'm looking for a tutor in Oakland, Berkeley, or San Francisco to help me fill in the gaps.

Ideally, I'd like to meet with a math student or professional for a half-dozen hour-long sessions that will help me understand terminology, notation, and concepts involved in linear algebra and (important!) how these apply to computer science domains such as data mining, search engines, document clustering, and other related topics. I'm particularly interested in being able to quickly understand and apply papers such as this or this.

I don't know how difficult this is for someone well-versed in the subject, so I can pay in food, beer, or US$. Amount negotiable. Write me at mike at stamen dot com.

Aaron Swartz recently moved to town, and I think he got hassled by some mission bike hipster, because he really hates it here despite having completely figured out the city in record time.

I'd like to nominate him for The First Annual John C. Dvorak Award For Outstanding Achievement In The Field of Attention Journalism. The souvenir award statuette (or "trollie") is pictured below:

The new Buzzfeed archive view is pretty damn cool. From their about page:

We automatically detect new buzz by crawling 50,000 of the very best web sites, blogs, and news sources. ... The moment we detect new buzz, it appears in a special terminal interface used by our editors. ... Finally we track the buzz as it spreads through word-of-mouth and blogs. Our trend pages link to the most interesting commentary, videos, news articles, and debate.

It's a new project from a bunch of our friends over at Eyebeam, and its worth to me was proven during my first pre-release peek, when I learned about New Rave. I'm mostly taken by the look of the thing, though. Dead-simple colors, dead-simple URL's, dead-simple typography, and a dead-simple logo that looks like it says "BuzzFeep" when you read it too quickly. The calendar-like archive view (below) is probably the coolest part. Buzzfeed performs basically the same function as Techmeme, but does it about 100x better through the intervention of a human editor choosing succint names, writing concise copy, and dispensing with the link-barf on a typical Techmeme page.

When I have a window open in Safari that contains an embedded YouTube clip, Safari's CPU usage shoots up to 20% - 50%. When I close the window, CPU usage drops to normal. What the hell is YouTube's flash player doing in there? I'm not playing the video, I've just got the player sitting there. It's not downloading the video, though it might be phoning home.

Oh, uh, we spoke at IDEA in Seattle last week. It totally rocked, I'll write about it soon.

I'm not entirely satisfied with the state of AWS access libraries out there so I wrote my own. Most of the ones that I have seen (e.g. boto) strive for completeness and object-orientedness. I mostly just need to push a few strings around, and dislike having a large collection of classes, custom errors, and super-documented API's to wade through when I need to get something else done. I'm aiming for the level of complexity found in python-memcached.

So these should be simple:

• AWS.tgz

So far I've only needed to use SQS and S3, so that's all that's represented there.

About five years ago, I first came across this article on Ishku'rs About Page (yes, that Ishkur):

The gods, say the Sumerians, came to Earth by way of the 12th planet Nibiru, whose highly elliptical orbit brings it within the inner solar system only once every 3600 years. Whether this means that the gods came to Earth from Nibiru or that they came via Nibiru is still hotly debated, but the fact persists that the gods, also known as the Elohim or "Lofty Ones", the AN.UNNA.KI, the Nefilim or "Those Who from Heaven to Earth Came" were, in the Sumerians' eyes, a technologically advanced race of beings.

It's a riveting story about the early (and modern) religions of man that ties up millenia of atomized history into a coherent, attractive picture. It's not important whether it's believable, because it's a good yarn. I read it and felt a similar desire to live in that world as I felt after reading C.S. Lewis's Space Trilogy. It's the kind of story whose emotional logic and psychological interest weighs more than accuracy or truthiness.

Adam tipped me off to the fact that Ishkur gets much of his material from Erich von Daniken's Chariots of the Gods. So I bought the book and read it, and although it contains pretty much the same information, it's just an awful, awful piece of writing for a variety of interesting reasons. First, Daniken has written and advocacy piece, and it matters to him that his readers actually believe that aboriginal rock paintings depict beings in space helmets or that the ancient Egyptians were too dense to stack rocks into pyramids. He uses words like "undoubtedly" to support his unfalsifiable arguments ("...drawings undoubtedly meant as signals for a being floating in the air ... Undoubtedly the Ark was electrically charged!..."), which instantly puts me on the defensive. Second, he selects evidence carefully, showing only rock paintings or statues that bear a superficial resemblance to the NASA spacesuits in use when the book was written. Third, he seems insensitive to long timescales, attention spans, and reserves of manpower. It may be overwhelming to imagine building a pyramid here and now, but Pharoah had a few resources at his disposal that Daniken lacks.

His greatest sin is overlooking the value of telling a fun story, the reason I got a kick out of Ishkur's piece and picked up Chariots in the first place:

So great was ISH.KUR's anger that during this period the Egyptians called him Prince Resheph, god of pestilence. He openly came and ravaged the people of Egypt with a kind of biological warfare so devastating that his wish was granted, and his people set free, just to get him to stop. But when he led them out of Egypt he learned that during their captivity they had been assimilated into Egyptian culture, worshipping Egyptian gods (EN.KI's family). This was a common tendency of ancient cultures-to readily adopt and owe allegiance to whatever deity was most powerful, led by the strongest army. ISH.KUR, who never had a permanent city and people to call his own, was extremely upset to find his people worshipping other gods. He had to find a way to make his people worship him and only him, by eliminating all other divine influences. It was there, in the Sinai desert, over a forty year span (so as to lead a new generation of followers, untainted by the years of polytheistic captivity) that he sowed the seeds of a monotheistic form of worship.

So skip the book, and check out the article. Also, it looks like Ishkur is also working on a new edition of the Guide to Electronic Music.

I went to my first Critical Mass on Friday, and I definitely plan to go back next month. The event happens in cities around the world during evening commute hour on the last Friday of every month. SF's local version is probably one of the bigger ones I have heard about, with thousands of cyclists showing up and occupying a few linear blocks of traffic along a route that varies each month. This month's route circled around SOMA, with detours up into the Transbay Terminal and back north across Market.

I noticed a slow leak in my rear tire around Mile #2, so I left the ride when it returned to Embarcadero BART. I wish I had been more prepared, because it seems to have ended up in the Broadway tunnel near the end.

(photo by McBomb)

I had my EC2 account activated today, and immediately ran off to follow the instructions to see how it works.

Apart from Amazon's baffling array of ID's, secret keys, user keys, certs, and other identifiers that need to be kept track of every few steps, EC2 is a winner. It does exactly what it promises, running virtual machine instances that can be instantiated and destroyed at will. The first one takes a few minutes to boot, and the process of making a machine snapshot is slightly laborious, but once the initial hurdles are clear, saved machine states start up almost instantaneously and are immediately available for use.

Combined with S3 for storage, this is a total winner. It's fortunate I just happen to be working on something that could benefit from 20 or so dedicated servers chugging away...

My costs so far: $0.31.

I've barely mentione iPods once in a few years of writing here, but this struck me as interesting. John Gruber has an article on Apple's pricing strategy in which he says:

The two Gartner researchers even lamented Apple's decision to discontinue the 1 GB nano, which they say could have been a nice mass-market item for around $99. Why not sell a 512 MB version, too? And what about 3 GB and 6 GB? And what about more colors?

...and:

...a 33 percent price reduction is not a small cut, and it would throw off their nice, even 2/4/8 GB for $150/200/250 pricing scheme.

I'm impressed that Apple designs their prices with the same attention to detail they use to design the rest of their stuff. I feel naturally more comfortable thinking about a price scale that looks as clean as "150/200/250" instead of one that requires a universe of useless choices. Apparently I'm not a rational economic actor.

From John:

The organizers asked me to design a flyer to hand out at the march. I took it as an opportunity to do something a little different from a typical flyer. ... The text is styled in the form of a redacted government document. It creates a parallel text that plays on themes of secrecy, coverup, and suppression of dissent, as well as seeing through the lies and reading what is erased.

I've been interested in calling cards for some reason, finding a bunch of cool material about them around the web: visiting cards etiquette, Crane's calling cards, and the famous scene from American Psycho.

Yesterday, Tom showed me a bunch of his new custom-printed cards from moo.com. Today they opened up their site with an offer of ten free, individualized cards based on your Flickr stream for all Pro account holders.

Who's an idiot for deleting his pro account now?

Next month, I'm going to be speaking in Seattle, at the IA Institute's IDEA2006. There have already been a few posts on the conference blog regarding my topic, and there are a bunch of other great speakers there, including Jake Barton, Alison Sant, Bruce Sterling, and Fernanda Viegas.

This weekend, the registration fee goes up.

Go!

I hit the tipping point with two pieces of technology today.

The first is subversion, a revision control package for software developers. I actually first heard about SVN approximately six years ago, at the same time I heard about CVS, my favorite until now. SVN was half-baked at the time, and I became an expert in CVS instead. I tried SVN a number of times over the years, but there was always something wrong enough with it (reliance on db, crappy merge support) to turn me off. This time, it's going to stick. The difference is that I'm experimenting with a project management tool called Trac, which is pretty much requires the use of SVN. The benefit is major enough that I'm finally convinced to switch entirely.

The second is LocoRoco, a playstation game smuggled into our office by Matt, who let me play a level.

The physics and graphics are amazing, and I love that the designers chose to ignore the normally-baffling profusion of buttons on the controller in favor of just three, and focused instead on getting the interaction flow just right. My favorite aspect of the game is the malleability of the character, who can be split or merged into a variable number of smiling orange blobs. The big one can jump higher, the small ones can slip through crevices, and together they form the basis of simple strategy. Very cool.

I never mentioned it, but I finished Patrick O'Brian's Aubrey-Maturin series earlier this year. It's a collection of 20 historical novels set in the British Royal Navy during the Napoleonic Wars (early 19th century). Gem turned me on to the series after we saw Master and Commander, and the books are absolutely brilliant.

The most interesting thing about the books as a whole is the arc of the plot(s). Very few of the books begin and end a single story. Instead, there tend to be three scales of plot: the mission that gives each book its title typically provides only a loose framework for microplot elements (battles, shipwrecks, etc.) of which there are dozens per volume. The macroplot moves from book to book, and concerns friendships, legal troubles, courtships, and marriages. Taken as a whole, the series is one long story spanning as many as two decades.

They're very much worth reading, if you take the necessary six-to-eight months to do so. It also helps to have the right dictionary.

From Andy, 1000 cars in a Trackmania run superimposed into one video. Hypnotic, reminiscent of simple particle systems, clouds of locusts, and herds of wildebeest filmed from a helicopter in equal parts:

We went on a ride through Golden Gate Park today.

I returned from DC last night, where I spoke at Adaptive Path's excellent UX Week event. I gave a 45 minute talk (my first solo thing) on Information Visualization, delivering a show and tell of projects that I've worked on, been inspired by, or just found really worth sharing over the years.

My slides are up at uxweek06.stamen.com, and Sam Felder blogged some notes that include helpful links to all of the projects I described.

I'm at Adaptive Path's UX Week in Washington DC right now. I'm noticing that Apple's Keynote software is leading to some excellent presentation visuals - it encourages big, simple slides with dark backgrounds. A lot of people are also using "one big image" slides, with full-bleed photographs or graphics. I like this. It's a great visual style for conference spaces, and really cuts down on the infoglut of your typical Powerpoint deck. So far, every session I've attended today uses this method.

On a tangent, DC's Metro subway is bizarro-world BART. The informational signs use Helvetica in exactly the same way, the ticket machines work almost identically, and the trains are similar as well. The two systems seem to be of the same vintage. Metro is predominantly "yellow-orange", BART is primarily "blue-silver". Both are great examples of Brady Bunch Futurism, a look that seemed outrageously advanced c. 1975, and now has a comfy, carpeted, conversation-pit vibe. I like this too.

(A brief record of products and services recently consumed)

1. Good bike: IRO Mark-V

   

   I picked this up from Montano Velo yesterday, and I'm thrilled. It's been a long time coming since I first chose it, but I'm still really happy with the decision. It looks different from what I expected, in that the rims are much deeper and the wheels are completely black. It looks like a spider. So far, the ride is quite comfortable. The brake placement and toe clips are taking some time to get used to, but the gear feels right and the size of the frame (which is 3cm taller than what I initially wanted) is also good.

2. Good art: Listening Post

   

   Worth the $8 price of admission to SJMA. This is confident, striking, immersive, funny, expansive work. Absolutely wipes the floor with We Feel Fine.

3. Good movie: Miami Vice

   

   Pretty much the perfect Michael Mann flick. I loved Heat, Collateral, and The Insider, and this one fit right in. He's really getting the hang of digital filmmaking, and is definitely the first director I'm aware of to use digital to create a new epic look. Don't miss the scenes of private jets streaking between clouds, various go-fast boats, Freaky Chakra's Blacklight Fantasy on the soundtrack, and amazing nighttime shots of Miami.

4. Good show: Survival Research Labs

   

   Part of the ZeroOne festival in San Jose. The big gold man sculpture (foily balls!) was a dud, and the show was without focus or point, but shit was loud, robots ate each other, 2x4's shattered, and everyone had a great time.

5. Good movie: Little Miss Sunshine

   As funny as 40 Year Old Virgin. This is the kind of movie that I want to see do well, so I will check on weekend box office numbers from time to time.

From GalaxyGoo:

I'm doing a Pyschology course and they had one of those charts that shows the number of sexual partners of average men and women. It never made sense to me that men could have more partners than women. I mean, who are they "partnering" with??? So I did a little data visualization, which makes things pretty clear. Don't worry, it's clean! 100 boys on the left. 100 girls on the right. Each gets assigned a number of partners based on a Kinsey survey. Then they...hook up.

Oooh, lower-right-hand corner, you skank!

(via Doug)

Krtek is a 1950's/60's Czech cartoon that I remember rebroadcast on Nickelodeon's Pinwheel, a 1980's Sesame Street knockoff. Each episode is short, and enormously sweet. The cartoons are wordless:

The first episode of the cartoon was narrated, but Miler wanted the cartoon to be understood in every country of the world, so he decided to use his daughters as voice actors, reducing the speech to short non-figurative exclamations in order to express the mole's feelings and world perception.

There are six episodes to be found on YouTube:

The Mole and the Swallow

The Mole and Ornaments

The Mole and the Snail

The Mole and the Coal

The Mole and the Box

The Mole and the House

Digg kicked Labs live 20 minutes ago. It's slow and intermittent, because it's being pounded right now. Anyway, go check it out - it's the first public release of our ongoing work with Digg over the past months, and we're proud and shaky.

After trying out a bunch of bikes last week, I decided on a completely different one:

IRO Mark-V

Adam actually recommended this to me about a month ago, and I've been considering it while checking out other options. Pretty much everyone I talked to about it likes IRO a lot, or knows people who do. Adam hooked me up with a friend who let me try his, and I like it enough that I put down a deposit on one this morning at Montano Velo.

Mine will have some modifications: front and rear brakes, bullhorn handlebars, and a flip-flop hub so I can try riding fixed gear if I want.

Stay tuned for photos of my first injury!

Just saw this now:

I made that logo a little over ten years ago, shortly after subscribing to SFRaves with my first UC Berkeley e-mail account. It existed in a few variations, starting as a series of laminate designs for finding other list members at parties, to a run of 30 T-shirts I hand-screened for a July 1996 FnF event.

For a college freshman, SFR was an insanely great way to discover the richness of San Francisco and the east bay. It was like a private tour of the undersides and hidden structures of the area, visited and reimagined for eight hour stretches every weekend. Beaches, warehouses, new office spaces, former office spaces, dance studios, and roller rinks were all fair game. The dot-com surge of the late 90's brought this to a head, increasing the number of partygoer hopefuls while simultaneously taking favorite spots out of play. The cultural and economic shifts that made the SFLNC necessary eventually doomed the vitality and energy of raving into just another industry. Now that I've lived here long enough to have a longer view of the area and a basic understanding of how all the streets fit together, underground events provide less revelatory context than they once did. The only solution I can think of is to move someplace new and learn a new street grid, or carve out whatever piece of my frontal lobes is responsible for spatial reasoning so I get to rediscover everything all over again.

I haven't been subscribed to SFR for a long time now, but I still enjoy seeing stuff like this surface from time to time.

I'm buying a new bike, and I'm not sure what to get. I want it to be single speed road bike, flip-flop hub a plus. I'm a sucker for simple-looking things, and will usually choose things based on lack of ornamentation. Half of my desire for a single-speed is a direct result of wanting a simpler, cleaner looking bike than the incorrectly-sized mass of cables and doodads I ride currently.

Here are the four I tried today, and a fifth that's out of my reach:

Cannondale Capo

Light. Frame feels very balloony: large, thin, sounds like a plastic kid-bat when tapped. Rides pretty well. Has this dopey "graf" design on the side that's kind of a turn-off, also 25% more money than the others.

Specialized Langster

Rides very, very well. Light, but solid in a way that the Capo was not. Strong favorite. Frame is painted like it's being marketed to skateboard kids, though the one on the Specialized site is a more tasteful black. Flip-flop hub is nice.

Bianchi San Jose

Feels heavy in comparison to the previous ones. Guy at the bike store says many of the employees own these, and that they're versatile and a great ride. Stylish in a 1980's tube sock sort of way, but I would probably buy narrower tires and clip pedals to replace the stock ones here.

Raleigh Rush Hour

Stylish dark grey. 57cm frame feels significantly smaller than the others - why? Pushing the pedals on this one makes me feel like I'm on point, instead of using the balls of my feet. Very uncomfortable, but if it's just the pedals this can be salvaged. Otherwise quite nice.

Jitensha Ebisu

I don't know why I walked into this store. These bikes are hand-made by master craftsmen in Berkeley, and the frames they have hanging in the windows are masterpieces of simplicity and strength. Easily 4x more expensive than the rest of the bikes on this page, like I'd even know the difference.

Oakland's Mandela Parkway, seen in Google Maps. This is the former site of the Nimitz freeway, a raised double-decker road that collapsed in the 1989 Loma Prieta earthquake. At 24th street, the boundary between two satellite image sources simultaneously shows the park during construction and after construction (run-up title for this post: Schrodinger's Park). Hopefully, a hint of things to come if Google opens up layers of historical satellite images in Maps.

Unexpected results from camera phones that scan the image field line-by-line, when used to photograph quickly-moving objects:

(Photo by MaciekSz)

Reminds me of an old project page on building digital cameras out of unused flatbed scanners, which featured the following gem:

An excerpt from The Power Broker, Robert Caro's 1974 tombstone on Robert Moses:

Underlying Moses' strikingly strict policing for cleanliness in his parks was, Frances Perkins realized with "shock," deep distaste for the public that was using them. "He doesn't love the people," she was to say. "It used to shock me because he was doing all these things for the welfare of the people. ... He'd denounce the common people terribly. To him they were lousy, dirty people, throwing bottles all over Jones Beach. 'I'll get them! I'll teach them!' ... He loves the public, but not as people. The public is just the public. It's a great amorphous mass to him; it needs to be bathed, it needs to be aired, it needs recreation, but not for personal reasons - just to make it a better public."

I'm reading this to fill in some backstory to Death And Life Of Great American Cities. Aside from being a dramatic account of urban renewal and destruction, the world of the 1920's and 1930's is a perfect context for similar "social architecture" taking place on the web, right now. Net Neutrality, User Generated Content, and Social Software all gain historical continuity from this story. Perkins's quote above throws an especially harsh light on the ink spilled over (Stamen client) Digg, which is one of a few examples used by writers like Nick Carr and Scott Karp to demean the quality of user-submitted Digg stories, MySpace profiles, and blog entries.

The interwar years are fast-becoming one of my favorite historical periods all-around, partially because so many of the lessons of that time are being forgotten as that generation passes on.

To paraphrase Malcolm McCullough, two things you don't say when you walk into a supermarket:

1. "I can't make somethig with fish, that's already been done."
2. "How am I going to fit all this stuff into my dinner?"

(Sketching wrap-up, on the topic of choice in hardware standards)

Last night we attended Digg's version 3 launch party, where Kevin showed two examples of our work to a packed house at Anu:

(Digg Incoming by Laughing Squid)

(DSOC by Laughing Squid)

I'm becoming an increasingly excited fan of Jan Chipchase's site, Future Perfect.

As I understand it, Jan is an ethnographer and researcher for Nokia. He travels the world, mostly Asia, photographing and documenting local practices relevant to Nokia's business. The site is a travelogue of tight, observant photography and short conjectures, and makes me curious what kinds of brilliance can be found in his actual reports.

A week ago I vented my dislike for ajax abuse. I got a few comments about that, including one from Ray, who recommended Safari Stand as a possible answer. I'm here to say that it works really well.

It's a bit of a mystery application, apparently written by a Japanese developer as a sort of browser swiss army knife, adding all kinds of navigation features to Safari. I've come across it before, but have stayed away because I generally prefer lean software. I've been able to ignore all of its features, except for site alteration:

Each domain listed on the left can be modified to enable or disable certain site features, and impose new behaviors or stylesheets. I have most of these sites modified to kill javascript, and a few also have font-size adjustments. It's been a significant change for the better. The New York Times and Wired are both now usable, and I've also disabled scripts on other sites that seemed to be doing a bit too much client-sniffing or slowing down normal interactions. Even Flickr benefits: what I thought was server slowness for the past few months turns out to have been Ajax lag. Photo pages load significantly more quickly, and the only drawback has been the appearance of useless "For a better experience, please upgrade to a Javascript-capable browser" messages on NYT and Flickr. The reason there are a bunch of Google's in the list is that I'm shutting out all of *.google.com, and selectively allowing certain google.com subdomains that seem worthwhile. Stand reads the list from top to bottom, and the first match wins.

So far, I've been taking a black list approach. Sites that annoy me get the axe. If the list grows especially long, I may switch to a white list and allow Javascript only on certain domains. In the meantime, this is working very well.

(photo of Mission High School by Ian Fuller)

We just bought an extra chair, and it wants to be filled:

Our busy San Francisco design and technology studio is ready to grow again. We're looking for a few people who can help us realize our vision: doing great work for the smartest people we know.

More info at stamen.com.

Update, Friday: Ray recommends Safari Stand, whose site alteration feature does exactly what I hoped for.

Rant on.

I like javascript, but its use by a few big sites (looking at you, nytimes.com and wired.com) makes me break out in hives. When I turn off javascript in Safari, both of these sites are screaming fast as I'd expect. When I have it turned on, both are excruciatingly slow, and simple actions (selecting text, clicking links) lead to baffling delays and beach balls. I haven't yet spent the time with Venkman to figure out why this is, but I have my suspicions. A lot of the scripts are coming from Google, Doubleclick, and other "strategic partners" checking in on my activity. Wired seems interested in where I'm from (see the "GeoIP" section of headerLayer.js), and The New York Times likes to know what text I'm selecting, and which links I'm following.

It would be ideal if sites like this put the Ajax crack pipe aside for five minutes and erred on the side of usability. It would also be nice if Safari's javascript implementation were faster, or threaded, or whatever.

Barring that, I have a feature request: per-domain javascript disabling. Javascript is now too useful and pervasive to be turned off entirely, but certain domains abuse the privilege and ought to be denied. Even a javscript on/off switch in the Safari browser chrome would go a long way towards helping.

Rant off.

Courtesy of CNN and Slashdot, another reason Net Neutrality is a good thing:

"I'm going to pay my $29.99 a month for access, and then I'm going to pay higher prices for consumer goods all across the economy because these Internet companies will charge more for online advertising."

Gem and I visited a mushroom farm last week for one of her classes. Each batch of mushrooms (shiitake, oyster, etc.) are cultivated in bagged blocks of sawdust and nutrients. Each plastic bag holds about a gallon of substrate, and has a small filter attached to it, allowing flow of air when the bag is sealed. These bags are single-use, and historically cost the farm 18 cents apiece. As a result of rising fuel costs (the bags are shipped from Texas), the price has gone up to 19½ cents apiece. Every day, they start 2,700 bags, so an extra $40 isn't going to break the bank. But I bet that extra cost is going to be passed on to the buyer, on top of all the other gas-price micropayments piling up across the rest of the items at the checkout counter.

I understand fuel costs—high demand, low supply, etc.—but these "preferred routing" fees proposed by AT+T's lawyersharks are purely artificial. They have no reason to exist, and they add no new capacity to the lines. As I understand it, performance across the board may actually drop, because of the additional overhead of looking at each packet to determine its priority instead of passing it off like a hot potato as quickly as possible.

We went to a party at the Oakland Columbarium last night. It's a warren of tiny rooms, connected by passages, each containing walls loaded with ashes of the deceased.

This photo was the best Google had to offer, by John Wiseman:

Our friend Kelly Porter was live-painting in the Meditation Chapel, and many other video projections, musical acts, and exhibitions were scattered throughout the space. The party was a little subdued overall. The space was gorgeous. We spent the first 20 minutes or so making our way up to the Garden Of Revelation, via Devotion, Bethel, and Rest. Each of the garden areas is vertically offset from the previous, which gives the entire space a stairstep feeling. These are connected by tiny staircases, and end in the Revelation, a tall open space with "cat walks" one story above ground level. I think the entire building is sunk into a hill, because we could see trees draped over the skylights in the uppermost rooms.

I felt as though we had seen it all, but looking at the floorplan afterwards it's obvious that a whole section of the building (centered around the Middle Chapel) remains unexplored.

This is an absolutely stunning building by Julia Morgan that deserves a return trip.

Update: Peter has a set of Columbarium photos on Flickr.

Very literal, readable comment stream display on Spreadshirt:

It's visually easy to separate comment text from attribution, and indiviudal commenters with a relationship to Spreadshirt (CEO, employee, etc.) are subtly called-out. The indentation on the left makes it possible to read down a series of comments without interruption by attribution slugs.

This is a piece of San Francisco healing around now-gone railroad tracks:

Dr. Ron Blakey, Professor of Geology at Northern Arizona University recently posted a series of images that track land-mass movement in North America over the past 550 million years.

I combined them into movies:

• Small, 1.8MB
• Medium, 3.2MB
• Giant, 4.7MB

Finally got the dots moving in almost-real-time, and made the site live: