Enter the Python module Shapely, which has a great touches function where you can see if two voting districts share a boundary, which I can make a requirement before associating a voting district with other districts.
Additionally, shapely does a lot of other great things that makes this process much easier. One of the big ones is that it allows you to create polygon objects, which has lots of helpful methods, such as finding a centroid, getting the bounds and area, etc.
So, I'm working on reworking pretty much everything to use the Shapely module. Stay tuned.