Next up, we are going to go through our starting points where each will take turns choosing a voting district that is closest to them. This draft of sorts is intended to cluster the voting districts together to build our congressional districts.
To do this, we are going to continually pick the closest voting district for each point until there are no more voting districts left to pick. As voting districts are picked, we will associate the voting district to the point by appending the point and its distance to the voting district to our big list consisting of each voting district and their center points (the value we used to measure the distance to the starting point). We'll also take the voting district away as an option by replacing the distance value in the matrix from the measure to 'xxxxx.' We'll know when we are done when all the values in one of the matrix rows consists of all 'xxxxx' values.
Here's the code I wrote with the usual verbose comments:
#Our list to iterate through the points, giving each a turn to get its closest voting district.
point_list = list()
#Next, until all the voting districts are picked, round robin through the points, designating the closest available voting
#district and removing that voting district as an option for the next pick. For the ones already found, mark out with xxxxx, which will also eliminate it from being
#found as the minimum for the next point check. Do this until all the voting districts equal xxxxx, which means they've all been designated to a point.
#we need a variable that'll be used to hold how many voting districts haven't been chosen.
choices_left = 1
#while there are still voting districts unspoken for...
while (choices_left > 0):
#I want to pull an item from a list of the points using pop, but make the pop or the list random...
#can do this with random.shuffle, then pop the last one out!
#when the list is empty, fill it back up with the number of points and shuffle that list.
if (len(point_list) == 0):
for e in range(0,len(start_points)): point_list.append(e)
random.shuffle(point_list)
point_choice = point_list.pop() #pull a point.
min_distance = min(matrix[point_choice]) #get the least distance found between a voting district and that point.
dist_loc = matrix[point_choice].index(min_distance) #figure out which voting district has that minimum distance.
coords[dist_loc].append([point_choice, min_distance]) #and apply the point and distance to that district as the points choice.
#then mark that voting district as chosen by changing the distance values corresponding to that district to xxxxx.
for f in range(0, len(matrix)):
matrix[f][dist_loc] = 'xxxxx'
#figure out how many voting districts have yet to be spoken for by counting how many in one line of the matrix do not equal xxxxx.
choices_left = len([g for g in matrix[0] if g != 'xxxxx'])
#print(str(point_choice) + ' ' + str(min_distance) + ' ' + str(dist_loc) + ' ' + str(choices_left))
Next, we've got to figure out what to do with this data. It would be great to build a map with it to see what it looks like, but I'm not sure exactly how to pull that off.
It looks like javascript and leaflet.js may be a nice option, but there's definitely a learning curve there for me. I'll research and see what I can do.
Monday, February 6, 2017
Thursday, February 2, 2017
Voting Districts Day 5: The district/distance matrix
Next, I'm going to build a matrix showing the distance between each voting district and the start points.
To figure out the distance between the two points, I'm going to attempt to use the pythagorean theorem (a² + b² = c²) where a equals the difference between the x coordinates and b equals the difference between the y coordinates. This'll be a measurement "as the crow flies."
Our point shared by the x and y axis will have the same latitude as one point and the same longitude as the other point to make a right triangle. To calculate the distance for the x and y lines, we'll subtract the latitudes and longitudes of the points between the centroid and the starting point.
(voting_district_x - starting_point_x)² + (voting_district_y - starting_point_y)² = (the_distance between the points)²
OR
math.sqrt(((start_points[a][0] - coords[b][2]) ** 2) + ((start_points[a][1] - coords[b][5]) ** 2))
Now I just have to do a double loop to build the matrix showing the distance between each voting district and starting point like so:
matrix = list()
for a in range(0, len(start_points)):
point_line = list()
for b in range(0, len(coords)):
point_line.append(math.sqrt(((start_points[a][0] - coords[b][2]) ** 2) + ((start_points[a][1] - coords[b][5]) ** 2)))
matrix.append(point_line)
And that should give me a nice matrix that I can run a draft through to assign voting districts to starting points! That's what we're going to work on next.
To figure out the distance between the two points, I'm going to attempt to use the pythagorean theorem (a² + b² = c²) where a equals the difference between the x coordinates and b equals the difference between the y coordinates. This'll be a measurement "as the crow flies."
Our point shared by the x and y axis will have the same latitude as one point and the same longitude as the other point to make a right triangle. To calculate the distance for the x and y lines, we'll subtract the latitudes and longitudes of the points between the centroid and the starting point.
(voting_district_x - starting_point_x)² + (voting_district_y - starting_point_y)² = (the_distance between the points)²
OR
math.sqrt(((start_points[a][0] - coords[b][2]) ** 2) + ((start_points[a][1] - coords[b][5]) ** 2))
Now I just have to do a double loop to build the matrix showing the distance between each voting district and starting point like so:
matrix = list()
for a in range(0, len(start_points)):
point_line = list()
for b in range(0, len(coords)):
point_line.append(math.sqrt(((start_points[a][0] - coords[b][2]) ** 2) + ((start_points[a][1] - coords[b][5]) ** 2)))
matrix.append(point_line)
And that should give me a nice matrix that I can run a draft through to assign voting districts to starting points! That's what we're going to work on next.
Wednesday, February 1, 2017
Voting Districts Day 4: Starting with Random Points
Now that we've got the center points for our voting district shapes, we need to figure out how to create central points where we can start our clustering.
There is a random library included with Python that I hope can do the job. It has a uniform function that will give me a random float between two numbers that I designate. I'll just loop through to generate these random coordinates based on how many clusters I want to build.
First, I want to make sure these starting coordinates fall somewhere approximately within the state of NC. So, I'm going to get the total min and max x and y axis values by writing ALL of the x and y points to two lists and pull a min and max from those lists to create a range for the random numbers.
import random
...
xmids = list()
ymids = list()
for x in range(0,len(shapes)):
for y in range(0,len(shapes[x].points)):
xmids.append(shapes[x].points[y][0])
ymids.append(shapes[x].points[y][1])
start_points = list()
xmax_o = max(xmids)
xmin_o = min(xmids)
ymax_o = max(ymids)
ymin_o = min(ymids)
From there, we'll use the random.uniform method through a loop to create whatever number of points we want to start with.
start_point_count = 12 for z in range(0,start_point_count):
start_points.append([random.uniform(xmin_o,xmax_o),random.uniform(ymin_o, ymax_o)])
And we now have our random starting points!
Next up, we are going to build ourselves a matrix consisting of the distances between the centroids of our voting district shapes and our starting points. From there, we will then figure out which voting district should be associated with each point by going through the voting districts draft style where each point will take a turn picking its closest voting district.
There is a random library included with Python that I hope can do the job. It has a uniform function that will give me a random float between two numbers that I designate. I'll just loop through to generate these random coordinates based on how many clusters I want to build.
First, I want to make sure these starting coordinates fall somewhere approximately within the state of NC. So, I'm going to get the total min and max x and y axis values by writing ALL of the x and y points to two lists and pull a min and max from those lists to create a range for the random numbers.
import random
...
xmids = list()
ymids = list()
for x in range(0,len(shapes)):
for y in range(0,len(shapes[x].points)):
xmids.append(shapes[x].points[y][0])
ymids.append(shapes[x].points[y][1])
start_points = list()
xmax_o = max(xmids)
xmin_o = min(xmids)
ymax_o = max(ymids)
ymin_o = min(ymids)
From there, we'll use the random.uniform method through a loop to create whatever number of points we want to start with.
start_point_count = 12 for z in range(0,start_point_count):
start_points.append([random.uniform(xmin_o,xmax_o),random.uniform(ymin_o, ymax_o)])
And we now have our random starting points!
Next up, we are going to build ourselves a matrix consisting of the distances between the centroids of our voting district shapes and our starting points. From there, we will then figure out which voting district should be associated with each point by going through the voting districts draft style where each point will take a turn picking its closest voting district.
Sunday, January 22, 2017
Voting Districts Day 3: Yet Another Package Change to Pyshp
Still having trouble with Fiona. So, I'm trying another package for reading shapefiles: pyshp.
Pyshp installs through pip without issue (as long as I do it as an admin). Hooray!
pyshp comes with the shapefile library, which reads a shapefile into a structure of lists and dictionaries.
What should I read? The NC Board of Elections has a shapefile that has all of the voting districts available on their FTP site.
And after downloading, we can read it like so:
import shapefile
vote = shapefile.Reader('ncsbe\\Precincts.shp') #creates an instance that has the lists of data we want.
shapes = vote.shapes() #lists of coordinates making up the shape for each voting district.
To figure out the center of the shapefile, I hope this isn't too simple:
What I should do is get the min/max for both the x and y, then average that. I'll put it all in a list.
coords = list()
for x in range(0,len(shapes)):
xmin = 10000000
xmax = -10000000
ymin = 10000000
ymax = -10000000
for y in range(0,len(shapes[x].points)):
xmin = min(xmin, shapes[x].points[y][0])
xmax = max(xmax, shapes[x].points[y][0])
ymin = min(ymin, shapes[x].points[y][1])
ymax = max(ymax, shapes[x].points[y][1])
coords.append([xmin, xmax,(xmin + xmax)/2, ymin,ymax, (ymin + ymax)/2])
If I want the metadata for each shape through the records method, this is how to do that.
recs = vote.records()
For now, I just care about the shapes and their distance relative to one another.
Pyshp installs through pip without issue (as long as I do it as an admin). Hooray!
pyshp comes with the shapefile library, which reads a shapefile into a structure of lists and dictionaries.
What should I read? The NC Board of Elections has a shapefile that has all of the voting districts available on their FTP site.
And after downloading, we can read it like so:
import shapefile
vote = shapefile.Reader('ncsbe\\Precincts.shp') #creates an instance that has the lists of data we want.
shapes = vote.shapes() #lists of coordinates making up the shape for each voting district.
To figure out the center of the shapefile, I hope this isn't too simple:
What I should do is get the min/max for both the x and y, then average that. I'll put it all in a list.
coords = list()
for x in range(0,len(shapes)):
xmin = 10000000
xmax = -10000000
ymin = 10000000
ymax = -10000000
for y in range(0,len(shapes[x].points)):
xmin = min(xmin, shapes[x].points[y][0])
xmax = max(xmax, shapes[x].points[y][0])
ymin = min(ymin, shapes[x].points[y][1])
ymax = max(ymax, shapes[x].points[y][1])
coords.append([xmin, xmax,(xmin + xmax)/2, ymin,ymax, (ymin + ymax)/2])
If I want the metadata for each shape through the records method, this is how to do that.
recs = vote.records()
For now, I just care about the shapes and their distance relative to one another.
Sunday, January 1, 2017
Voting Districts Day 2: Wrong Python Packages?
I think that I'm going to switch up and use some different packages for reading this spatial data. In order to use the SciPy stuff, I still need more packages and those packages require some unorthodox installation methods. It includes a ton of stuff I'm not sure I need. So, I'm going to do this one package at a time. Well, two in this case.
It looks like Shapely and Fiona can do what I want initially, which is to read shapefiles and plot them.
When I try to use pip for the install ("pip install Fiona"), it doesn't work. It says I need Microsoft Visual C++.
So, after searching the internet, I discovered that I have to install the packages via a wheel. A plethora of these wheels can be found here at this University of California - Irvine website here.
You also use pip to install these wheels; you just have to download them into the same folder first. Also, I have to run the command prompt as an administrator via a right click when starting the application.
Shapely seemed to work like a charm when attempting the old "import shapely" line in Python. Fiona had other ideas.
Looking at Fiona's documentation, she requires a GDAL package, which was also available on the same UC Irvine page that had the other wheels. I installed that one and was finally able to run "import Fiona" successfully.
So, next time, we'll see about reading the shapefiles.
It looks like Shapely and Fiona can do what I want initially, which is to read shapefiles and plot them.
When I try to use pip for the install ("pip install Fiona"), it doesn't work. It says I need Microsoft Visual C++.
So, after searching the internet, I discovered that I have to install the packages via a wheel. A plethora of these wheels can be found here at this University of California - Irvine website here.
You also use pip to install these wheels; you just have to download them into the same folder first. Also, I have to run the command prompt as an administrator via a right click when starting the application.
Shapely seemed to work like a charm when attempting the old "import shapely" line in Python. Fiona had other ideas.
Looking at Fiona's documentation, she requires a GDAL package, which was also available on the same UC Irvine page that had the other wheels. I installed that one and was finally able to run "import Fiona" successfully.
So, next time, we'll see about reading the shapefiles.
Monday, December 19, 2016
New Python Project: Is there a way to build nonpartisan voting districts?
Gerrymandering seems to make the news every couple of years and every couple of years I wander if there's a solution to the problem. It seems that somebody should have figured out how to make fair and equitable congressional districts by now, right?
I'm a little curious if something like that is possible using some geospatial analysis. What are the roadblocks and what's stopping somebody from pulling this off? Who knows? It's an excuse to learn more about Python and its application to mapping and such.
How much experience do I have with Python and geospatial analysis? none! Well, I've done a little, but with esri's arcpy Python module. This should be WAY harder, but a fun challenge.
Poking around the internet, it looks like Python needs some packages to do what I want (or at least get me started). A place called SciPy can hook me up.
They allow installing via pip, which I've done a couple of times in the past. They even tell me exactly what to type. So, to Command Prompt we go and we'll do the install.
Easy peasy. Next time, we'll see what we've installed and how it can help us.
I'm a little curious if something like that is possible using some geospatial analysis. What are the roadblocks and what's stopping somebody from pulling this off? Who knows? It's an excuse to learn more about Python and its application to mapping and such.
How much experience do I have with Python and geospatial analysis? none! Well, I've done a little, but with esri's arcpy Python module. This should be WAY harder, but a fun challenge.
Poking around the internet, it looks like Python needs some packages to do what I want (or at least get me started). A place called SciPy can hook me up.
They allow installing via pip, which I've done a couple of times in the past. They even tell me exactly what to type. So, to Command Prompt we go and we'll do the install.
Easy peasy. Next time, we'll see what we've installed and how it can help us.
Monday, November 21, 2016
PI Critter Cam Day 16: Better Pictures
I set up the critter cam yesterday afternoon after adjusting the setup.py file in the pi-timolo folder to rotate the pictures 180 degrees and take pictures at a 1080 x 1920 resolution. Luckily, a bird came by to test out the new settings.
The picture looks MUCH better! However, we have a couple issue where it's taking and keeping a TON of pictures and it's overwriting all of the previous pictures.
So, within the setup.py file, I'm going to change the threshold setting to 50, which should make the picture taker less sensitive. I'm also going to change motionnumon to False so it'll use a date/time stamp to name the file. I'm also change motionnummax to 0 in case it's using that setting as justification to delete stuff out there.
We'll set it out again tomorrow morning to see how it does. This time, I'll try setting it on the ground.
Subscribe to:
Posts (Atom)

