That's the line we use around the office whenever the subject of HTML and canvas comes up - we use Adobe Flash for most everything now, but we don't expect that situation to last forever. The work done by Mozilla on Gecko and Apple on WebKit is one possible future for online design and visualization, and it's turning slowly to face us right about now.
A few developments during recent years have brought us here.
The web is going to win, filling the market niche where Flash and other similar technologies found their hold. And John's little project can hopefully act as a great catalyst to take us there. Thanks, John!
Finally, as we work on increasing amounts of public, government, and civic work, we're mindful of the relative opacity of Adobe's various transmission formats, as noted by Sunlight Labs in Adobe Is Bad For Government. They're talking about PDF files in particular, but I liked this excerpt from an Adobe open government white paper: "Since the advent of the web, an entire infrastructure has evolved to enable public access to information. Such technologies include HTML, Adobe PDF, and Adobe Flash technology." Not wrong, necessarily, but then again Froot Loops is also part of a balanced breakfast so there you go.
Anyway, interesting times.
Usually when I experiment with a new technology like this one, it's nice to revisit an old problem or two, to see if some new light can be shed on it with a new tool. Coincidentally, while John Resig was releasing Processing.js in May 2008, I was busy working on some ideas for flea market mapping. I had recently found GDAL, a toolkit for manipulating digital raster maps, and rubbersheeted a collection of old Oakland road and real estate maps to match the one true cartographic projection. The annoying bit in the middle of this process is a three-way dance between a graphics program, a web browser, and a text editor for the collection of ground control points used to warp the image:
One of the simple things that canvas can do really well is transform and clip bitmaps, so it's possible to take an image file, turn it into a set of triangles, twist and bend them, and rapidly render the result. Cassidy Curtis clued me in to the linear math behind these operations many years ago, it really does just turn out to be a bunch of arithmetic.
This will be important later.
A thing about triangles is that there are some good algorithms for taking a bag of points and making sets of triangles out of them. Delaunay triangulation in particular is a simple, aesthetically pleasing mesh useful for dividing up a space defined by known points. Fred Scharmen has been writing about manual performance of graphics algorithms, and he has an excellent recent post on drawing the Delaunay triangulation and the Voronoi diagram.
Amsterdam designer Jonathan Puckey has been exploring a similar vein with his triangulated portraits, though sticking to the computer all the same.
Both maps can be panned around, and zoomed with the mousewheel. Move the circular control points around on each side to see how they change the visual display. I've tested this with Firefox and Safari on a Mac, and the resulting interaction is just a little bit snappier in the latter. There's a distracting visual flickr in both, which I'm sure could be dealt with by double-buffering the canvas or some similar optimization.
There are a few things here I really like about the canvas interface. It seems to use a stack model for the drawing state (with save() and restore() instead of the more traditional, cryptic push() and pop()) to save things like clipping paths and coordinate transformations. This makes it possible to load the image pyramid tiles into the warped view asynchronously, throwing a transformation and clipping triangle into an array with a bunch of closures and callbacks to blit out pixels when images are loaded. It's not particularly encapsulated or object-oriented, but it does keep the pixel manipulation model working consistently. Something that canvas definitively lacks compared to an environment like Flash is any concept of object identity - once you draw out a shape, it's promptly forgotten and unavailable for things like mouse hit tests. This is what SVG is for I gather, and I hear that the original Processing environment has some of the same limitations.
One thing I've also found is that the current implementations of this technology are complete CPU hogs, as measured by the readiness of my Macbook to kick its cooling fan into high gear - canvas-only interfaces like Ask Ken devour cycles.
I certainly believe it's a possible improvement over the interaction model of the current state-of-the-art in online map warping, Schuyler Erle and Tim Waters's New York Public Library Map Rectifier. God bless those guys but the modal interface with server round-trip for retrieving a preview can get a bit clumsy.
I'm fervently hoping that an approach like this triangulated mesh can bump the interactiveness and usability of the NYPL map project, whose output could be so incredibly useful to an old map collection.
More broadly, I'm finding that the plugin-free development route might be a good one to pursue for real, mature interactive work. The rendering engines behind Safari and Firefox and becoming ubiquitous on desktop and mobile devices, and if you're willing to ignore the increasingly-irrelevant Internet Explorer and the occasional cooling fan overdrive you should be golden. Tom uses an informal measurement of the number of animated objects an environment can simultaneously throw at a screen, and canvas is definitively catching up to Flash quickly, while retaining the critical benefit of a fast, seamless start, something that Java has always lacked.
http://blog.klokan.cz/2009/10/raster-map-reprojection-warping-with.html seems to be heading in an interesting direction too. Nice work!
Thanks Mike. I love starting the week with the future staring me in the face!
Nice writeup! I think of Canvas as just a bag of bits; its basically just a scriptable image tag -- you have to keep track of everything yourself, which is great when you don't _want_ to keep track of things so you can get raw performance without overhead. SVG makes more sense with what Protovis is doing, since they need hit tracking and object tracking to do their graphs in general. Nice demos with the maps BTW! You should put the source up for folks to download and play with. Best, Brad
Thanks guys! Howard, I've been following Klokan's work a bit, I think it's interesting that he's using proj4js and a pixel-by-pixel model. It's not what I'd call fast but the results are beautiful. Brad, I agree that Protovis really needed to move to SVG. I do like that they've hit both sides of the aisle and settled on an optimal target. The source code for all this stuff is now on Github: http://github.com/migurski/canvas-warp Amenity, it's nice to see you too. =)
This is a really excellent, smart post! I have a question, since I'm such a beginner. I don't quite understand how widely supported these technologies are (or will be). In particular, how much of this works on common versions of Internet Explorer? I've done some reading on this, but it's obviously complicated (like it's not supported unless you download something from Google? but will people do that?) I'd appreciate hearing your take on this.
Oh, I should add... on Internet Explorer 7 on Windows, I get errors on Protovis and on the canvas warp example. I desperately want to use this technology myself, but if it's not in a relatively recent IE, I'm worried.
Mr. Beginner, I'm pretty certain that none of this will work on common versions Internet Explorer. According to http://en.wikipedia.org/wiki/Canvas_(HTML_element) there is a plugin for IE that supports canvas, which is at least a similar situation to Flash, minus the install base of Adobe's plugin. To be totally honest, IE has been such a consistent headache for so many years that I simply ignore it when working on this stuff. If MS feels the need to get on this particular boat then I will applaud them, but it's definitely not something I'm willing to chase down.
Oops, that link should have a closing paren. Here: http://teczno.com/s/664
Thanks! This is very helpful. And alas, newbie though I am, I've already experienced a few Explorer-induced migraines. How do your clients react to non-IE-compatible work? Can we perhaps hope for change in attitude that lets us ignore Explorer?
Last week I had similar problem. We got a SVG map that wasn't in Mercator Projection. We want to add place marks based on geo position so we need to transform it somehow. Looks like Delaunay triangulation might be a way to go.
Marcin, hope it works out for you! There are a wide variety of tools in the GDAL suite that you might find useful as well, especially if the SVG map is in another known projection like Lambert or Albers.
hy in surfing on the web i find your blog and also can see that you are on FFFFound. So i research also an invitation. Thus if you want ... nico
Sorry Nico, I haven't seen a new Ffffound invite in more than two years.
Sorry, no new comments on old posts.