Michal Migurski's notebook, listening post, and soapbox. Subscribe to this blog. Check out the rest of my site as well.

Aug 22, 2004 2:56am

temporary troubles with In The News

I discovered today that Google had made some small tweaks to the HTML structure of their news page. It caused an interruption in my ability to scrape their site between the afternoon of August 20th and the morning of August 21st. Hopefully this won't be noticeable by tomorrow as my fix works its way through various levels of caching, but it does highlight the difficulties of working with capricious methods like page scraping. I'm currently looking at the feasibility of doing a similar visualization investigation with Open Secrets, and troubles with unexpected modifications to data formats underscore the usefulness of web services with stable API's. "Small pieces, loosely joined" —Flickr gets it, even Google gets it with their normal search service. Here's hoping that Google News moves out of beta soon.


Sorry, no new comments on old posts.

August 2018
Su M Tu W Th F Sa

Recent Entries

  1. planscore: a project to score gerrymandered district plans
  2. blog all dog-eared pages: human transit
  3. the levity of serverlessness
  4. three open data projects: openstreetmap, openaddresses, and who’s on first
  5. building up redistricting data for North Carolina
  6. district plans by the hundredweight
  7. baby steps towards measuring the efficiency gap
  8. things I’ve recently learned about legislative redistricting
  9. oh no
  10. landsat satellite imagery is easy to use
  11. openstreetmap: robots, crisis, and craft mappers
  12. quoted in the news
  13. dockering address data
  14. blog all dog-eared pages: the best and the brightest
  15. five-minute geocoder for openaddresses
  16. notes on debian packaging for ubuntu
  17. guyana trip report
  18. openaddresses population comparison
  19. blog all oft-played tracks VII
  20. week 1,984: back to the map