Hack the Survey

2017 Australian Marriage Law Postal Survey - Results Map
I had been looking for a good coding project for a while, to try and build some confidence before I attempted a hackathon, when a friend posted a link to a great map based representation of the  Australian Marriage Law Postal Survey results.  It was a nice clean map that provided a mouse-over pop-up of the yes/no percentage for each electorate which I found it really interesting, but I also wanted some more information.  Information on things like the number of voters, the member holding that seat, the party they are from, what suburbs where in each electorate, and maybe even some census data.  Thus the project was formed.

As always, the 80:20 rule was proved true when I got the map up with the basic info the following day, but being a perfectionist (with a small child and a few other project on the go), it has taken me quite a bit longer to show it off but here it is:
2017 Australian Marriage Law Postal Survey - Results Map
(The site is hosted on shinyapps.io so hopefully it doesn't run out of server time)

So I'm actually hoping it isn't finished, there are a number of other thing I would like to continue adding:
  1. More census data sets
  2. Improved mobile/touch device support
  3. Change the layout
  4. Improved loading speed
  5. Plot formatting and grouping to improve readability
but these things can continue to happen slowly over time.

Just very briefly, I know that a lot of people have been hurt or angered by the survey so I hope this isn't seen as taking side or bringing up bad memories, my intention was to build a tool for examining the data and inspiring people to do more with their data.  So along those same lines, if people are commenting, it would be great if you could keep it about the tech, the data and the analysis.  Thanks!

Technical Details

For those who that are here for the technical details and challenges, here's what you need to know...


The site is built with Shiny, a web app language based on R developed by  RStudio which can run R code in the background to generate an html page a browsers can view.  The actual layout code is like a cross between R, html and javascript but it is relatively easy to learn, particularly if you have some experience in R and html.

The map is powered by leaflet, a javascript plugin for R and Shiny which allows you to display map tiles, both hosted from an external wms server (including a number of built in base maps), or rendered from your own spatial data.  Leaflet is really easy to use in Shiny but I ran into a few issues along the way.
  1.  Leaflet can be very slow to load files, particularly complicated polygons.
    1. To get around this I simplified my polygon layers as much as possible (you will see the don't perfectly match up in places).
    2. Even with this the suburb layer was very slow to load so I set up a button to load it separately for those that wanted to use it.
  2. Census data can be hard to wrangle into a useful format for plotting
  3. The help files for Leaflet are not always easy to understand when you don't have a background in coding, like how to read button clicks and switch layers on and off.
  4. There are some limitations in Leaflet that I'm still hoping to overcome.
 I'm hoping to update this article as I progress the site and I also hope to write some help articles on the challenges I found and (hopefully) overcame in building the site.  Let me know if you have any questions or a topic you would like me to start with.


Comments

Post a Comment

Popular posts from this blog

Victorian Property Overlay Map

Image
My wife and I are currently house hunting and this will be the first house I've ever bought so I'm a little nervous about not messing it up (perfectionist and all that). So when we find a property we're interested in I dig into the details by checking for easements, looking at the drainage and checking planning overlays since I know a little about these things from my time in a Local Government engineering department many moons ago. Anyway it turns out that a lot of these things are available as open data, for the whole of Victoria! So to make my life easier I put together a little web app that also works on mobile devices for when you are actually at an inspection. Victorian Property Info Having recently built the Cycle Melbourne map , this one was super quick to put together. I used the same layout, almost the same background maps and just added 4 WMS map layers from land.vic.gov.au . The Leaflet Extras package also allowed me to add a search function and a...
Read more

On your bike... to where?

Image
The great bike rush of 2020 is in full swing! COVID-19 restrictions means bike shops are sold out and there's a huge increase in cycling everywhere... or so media (social and otherwise) would have you believe, but I haven't seen any details on what is actually happening. So as a cyclist and data scientist I thought I would try and find out. I wrote a deconstruction of the reported 700% increase in cycling in Cycling, the new COVID-19 side effect? where found some serious sampling bias issues and proposed a more realistic results of 10-150% increase on weekends and a reduction on weekday at most location. While far more robust (in my option anyway) it still did not provide extensive coverage and did not look at changes in where people were riding. So here we are for round 2 so lets try and make it slightly scientific. My hypothesis (theory of what's happening) is that over the lockdown period: weekday cycle counts will be lower and weekend counts remain unchanged or higher,...
Read more

Everyday IoT: Tank Water Levels

Image
A few years ago I moved house and became the owner of a nice backyard with a garden.  However with early arrival of summer, I quickly discovered the challenge of keeping everything alive!  As holidays and heat-waves approached it became pretty clear we needed an automated watering, and one that could keep track of tank levels to avoid running the tank dry and damaging the water pump. Add to that the fact I'm always out to save a buck, I was inspired to do a little IoT project... Final set-up, sorry about the spoilers I iterated on this idea for quite a while with ideas ranging from full computer controlled watering system to something much simpler.  In the end I decided my requirements were: Have a watering system with an automatic timer Be able to switch off the pump remotely  Be able to check tank levels remotely I eventually realised that two of these would be much easier (and probably no more expensive) to buy as off-the-shelf components but there was stil...
Read more