Google Location History Part III

While I was researching more ways to visualize my Google Location History, I ran across Beneath the Data’s excellent post on exactly that. I thought I’d start with trying to replicate what Tyler had done (because his figures are way prettier than the ones I’ve made thus far), and then doing my own thing from there. Little did I know that this would send me down some giant GIS/shapefile/geography rabbit hole.

I have to be honest, my main takeaway from the Beneath the Data post was that you could download shapefiles and use those to make your figures instead of the default world stuff from Basemap that I had been using. I’ll also admit that, until this point, I thought the Basemap’s drawmapboundary(), drawcoastlines(), drawcountries(), etc were super cool. Not as cool as using some super detailed and specific shapefile, though!

I immediately went to download a bunch of shapefiles of San Francisco, California, etc. I got obsessed with Shapefiles. I saw a bunch used for ecological studies of the bay, but (though I was sorely tempted) I did not download those. Unfortunately, this is where I hit my first snag. Apparently, Seattle uploads shapefiles with WGS84 coordinates. San Francisco, alas, does not. Anywhere. Ever.

There is a boundless sea of vector files available for SF, but no WGS84 coordinates to be had. I researched for an embarrassing number of hours of how to convert to WGS84 and trying to figure out if they were calculations I could automate, and how exactly the x,y coordinates for shapefile vectors were calculated. I even installed QGIS because somebody said that you could save a layer as WGS84 through there. Well, I couldn’t get that to work. Maybe it is due to my inexperience with GIS and shapefiles, or maybe it is a function that QGIS doesn’t have anymore. I don’t know.

Finally, I found out that gdal has a super easy way to convert shapefiles to GeoJSON. It is literally just one command. PHEW. At last, I could download any Shapefile I wanted, and instantly convert them to get WGS84 coordinates if they didn’t already have them. I went with this San Francisco Zip Codes shapefile, and converted that.

Of course, immediately after I figured out that gdal has this functionality, I found a California County shapefile that already had it in WGS84 coordinates, no conversion necessary. ISN’T THAT ALWAYS THE WAY.

Anyhow, I spent so much time learning about maps and shapefiles, that I haven’t yet done anything beyond simply replicating what Tyler did, but with my own data. But now that I’ve gotten all that out of the way, making the visualizations that I want will be the next step.

My location density for San Francisco between 9/20/2013 and 5/22/2016

My location density for San Francisco between 9/20/2013 and 5/22/2016

Excerpt from my location density in California Counties between 9/20/2013 and 5/22/2016

Excerpt from my location density in California Counties between 9/20/2013 and 5/22/2016

My older post Google Location History Part II can be read here.

About Steen

Steen is a nerdy biologist who spends a lot of time trying to cultivate Chloroflexi. When she isn't working, she likes to draw comics, play video games, and climb.
This entry was posted in JSON, Programming, Python and tagged . Bookmark the permalink.

6 Responses to Google Location History Part III

  1. Pingback: Results for week beginning 2016-06-06 | Iron Blogger SF

  2. That location density chart is so cool.

  3. Jeffrey Blumenthal says:

    From the perspective of a geography student (i.e. amateur – take with grain of salt): It’s probably not necessary to do the conversion, if I’m understanding your dilemma correctly, which is a coordinate system issue, not a data format issue. [This was a great opportunity for me to brush up on the difference between a datum (i.e. mathematical description of the earth as a spheroid relative to a central point, e.g. center of the earth) and a geographical coordinate system (GCS), which is a location reference system based on a datum, different than a project coordinate system. Thanks!]

    The SF ZIP code file is based on NAD1983 (North American Datum from 1983). I don’t know what Google uses — probably WGS1984 (World Geodetic System of 1984), I’d like to look into that further. Fortunately, GCS coordinates based on the WGS1984 datum and NAD1983 datum are basically identical/interchangeable, which should be visually apparent if you display them both on top of each other.

    Good resource/description here:
    http://resources.arcgis.com/en/help/main/10.1/index.html#/Datums/003r00000008000000/

    • Steen says:

      That is super helpful, I didn’t even know enough to know what the zip code file was using, so I didn’t even know where to start searching for how to convert it! While converting it to geoJSON got the job done, I’ll admit that it was roundabout, and a more elegant solution would be a direct conversion between NAD1983 and WGS1984. I only ever knew about WGS1964 before, since that is what Google uses.

      Thank you so much for looking into that!

  4. Pingback: Google Location History Data Part IV | Certainly Strange

  5. Tyler says:

    Hey, cool to see someone take that post and run! I too ran into whacky issues with WGS84 and shapefiles. Glad to hear about gdal – I had no idea! Super useful, I wish I’d known earlier.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.