Michal Migurski's notebook, listening post, and soapbox. Subscribe to this blog. Check out the rest of my site as well.

Aug 21, 2004 10:56pm

temporary troubles with In The News

I discovered today that Google had made some small tweaks to the HTML structure of their news page. It caused an interruption in my ability to scrape their site between the afternoon of August 20th and the morning of August 21st. Hopefully this won't be noticeable by tomorrow as my fix works its way through various levels of caching, but it does highlight the difficulties of working with capricious methods like page scraping. I'm currently looking at the feasibility of doing a similar visualization investigation with Open Secrets, and troubles with unexpected modifications to data formats underscore the usefulness of web services with stable API's. "Small pieces, loosely joined" —Flickr gets it, even Google gets it with their normal search service. Here's hoping that Google News moves out of beta soon.
October 2017
Su M Tu W Th F Sa

Recent Entries

  1. blog all dog-eared pages: human transit
  2. the levity of serverlessness
  3. three open data projects: openstreetmap, openaddresses, and who’s on first
  4. building up redistricting data for North Carolina
  5. district plans by the hundredweight
  6. baby steps towards measuring the efficiency gap
  7. things I’ve recently learned about legislative redistricting
  8. oh no
  9. landsat satellite imagery is easy to use
  10. openstreetmap: robots, crisis, and craft mappers
  11. quoted in the news
  12. dockering address data
  13. blog all dog-eared pages: the best and the brightest
  14. five-minute geocoder for openaddresses
  15. notes on debian packaging for ubuntu
  16. guyana trip report
  17. openaddresses population comparison
  18. blog all oft-played tracks VII
  19. week 1,984: back to the map
  20. bike eleven: trek roadie