Mapping Shapefiles from the State of North Carolina

I've been looking forever on how to use shapefiles originating from North Carolina for making maps.  Normally, I'll get crazy latitude and longitude coordinates if I plot the shapefiles using the default parameters through R's rgdal package.

Tonight, I stumbled upon this StackOverflow post, which shows that TWO functions are needed in order to fully utilize NC shapefiles: readOGR and spTransform.

I've always just used readOGR, which works fine for shapefile originating from other places, like the US Census bureau.  I've always had problems with files from my state, however, until I came across the aforementioned StackOverflow post.

Here's what I put into R to make a simple map of all roads in Wake County:

roads <- readOGR("c:\\data\\poly\\wake_streets\\streets.shp","streets", p4s = CRS("+proj=lcc +lat_1=34.33333333333334 +lat_2=36.16666666666666 +lat_0=33.75 +lon_0=-79 +x_0=609601.2199999997 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=us-ft +no_defs"))
roadmap <- spTransform(roads, CRS("+proj=longlat +datum=WGS84"))
plot(roadmap,axes=T)
title("Roads in Wake County, North Carolina")

This'll produce a map like so:

If you want to try this at home, you can get a bunch of shapefiles from North Carolina's State Board of Elections FTP site that seem to work with this code.

Now that this mystery has finally been uncovered, my mapping options are much improved!

Getting Geocodes through R and Google's Web Service

Part of my new job as a Data Integration Analyst is learning how to study and manipulate data.  So far, I've really enjoyed this new challenge and I love having the opportunity to learn something new.  

I learned pretty quickly that R is a pretty popular programming language within the realm of data and analytics.  By itself, R can perform some complex data analysis. However, packages provided by other R enthusiasts can be loaded into the R interface to make it more powerful.  I've spent the last few months getting more familiar with the language and additional packages and learning to appreciate it.  Although I still have a lot to learn, I can already see that R can do a lot of really cool stuff.

One aspect of analytics that I've been particularly fascinated with involves analyzing data through geography.  R has a lot of packages that make this pretty straightforward.  The ones I've seen so far are great, but, in order to map a specific place, you need geocoordinates (latitude and longitude points).  Providing just an address to R and one of these mapping packages won't do.

I really want to map some data regarding voters in my home county, Wake county, North Carolina.  I think I figured out how to do it.

Google provides a free web service that allows you to collect geocoordinates for any address. All you have to do is provide Google with a residential address through a URL.

R has a function that allows you to collect data through the web.  It's as easy as this:

getweb <- url('http://maps.googleapis.com/maps/api/geocode/xml?address=1600+Pennsylvania+Avenue,+20500&sensor=true')
getaddress <- readLines(getweb)
close(getweb)

I've just requested the geocoordinates of the White House, placed the results in another object, then closed the connection with Google.

Google returns the data in an XML string, which is now in my 'getaddress' object.  Google can also return JSON, but R has a package that can interpret XML for you. Once you install the package, you can collect the coordinates from the XML like so:

lng <- xmlValue(getNodeSet(xmlParse(y),'//result//geometry//location//lng')[[1]])
lat <- xmlValue(getNodeSet(xmlParse(y),'//result//geometry//location//lat')[[1]])

You now have coordinates!  Using one of R's available mapping packages, you can plot it like so.  

This simple map was created using one of the easier of R's maps packages to create a map. Here's the process:

map('usa',bg='lightblue',col='tan',fill=T)
points(lng,lat,pch='*',cex=10,col='red')

This is really just a glimpse into the world of mapping through R.  There's a ton of resources out there that allow you to map all sorts of regions, locations, boundaries, and landmarks. 

The possibilities are endless.

NOTE:  Google is very generous to provide geocoordinates for free.  However, they do limit the number of daily queries for each person to 2500.  

Best Football Conference for Getting Drafted

A few weeks ago I was talking to my coworker about how there's a perception that SEC teams dominate college football.  If this were true, I speculated this may be due to better recruiting.  Winning schools typically recruit better.  Recruits want to play for winners. Teams with more exposure and evidence of success are more likely to have their players get drafted into the NFL.

However, is this really the case?  Do players in winning conferences, specifically the big, bad SEC, get drafted higher than players on "weaker" conferences?  Does the best recruiting conference perform the best on the field?

According to scout.com,  the SEC conference is, in fact, typically the best recruiting school.

As you can see from the chart above, the SEC typically gets more top 100 recruits than the other big six conferences (ACC, Big Ten, PAC 12, Big 12 and Big East).  The SEC's only been bested ONCE since 2002.

Does recruiting equate to wins?  They're pretty good each year, but it's not as clear cut as you would think.  However, the SEC is always one of the more highly ranked conferences when the dust settles at the end of the season, according to the Associated Press rankings.

You can see here the SEC teams usually fare better with top 25 votes than schools in other conferences.  However, there have a few years where other conferences have outperformed SEC schools.

Does the SEC pay dividends to the recruits who commit to their schools and help collect top 25 rankings when compared to the other conferences?  NFL teams do draft more SEC players than the other conferences in the majority of years since 2000, but it's certainly not a safe bet.

 This chart show that both the ACC and Big 10 have had more draft picks than the SEC on two occasions since 2000.  In most years, the margins between conference schools is pretty narrow.

So, Recruits fare pretty well on NFL draft day when committing to SEC schools, but it isn't necessarily always THE safest conference to bet on.  The SEC does the best job recruiting, but it doesn't ALWAYS equate to being the dominate conference every year.

Fixing a Dripping Delta Ball Faucet

Almost immediately after replacing the drain stop in the kids bathtub, the faucet started dripping.  Grrrr.

The tub has one of these types of faucets.  It's called a ball faucet and looks like this:

Instead of continuing to push down as hard as humanly possible on the handle, trying in vain to stop the dripping, I decided it would be a better use of my time to just fix the darn thing.  After a little bit of research, I was ready to tackle this small project.

According to most of the home improvement websites that I visited, dripping faucets are typically caused by worn out washers and other rubber parts.  This seemed to be a pretty universal reason for any type of faucet assembly (judging by my 30 minutes of internet research).

Here's a drawing of a ball faucet layout from Meriam Webster (good idea for them to branch out from being just paper dictionaries! You don't see too many of those anymore.)   The handle's different, but the guts appear to be the same.



With this knowledge and schematic, I took the faucet handle apart and checked out the rubber parts.

From this previous project, I had all the replacement rubber parts.  Since I was taking it apart anyway, I figured that I would just go ahead and replace the rubber pieces.

First, turn the water off to the faucet!  If there's no handle, there's nothing to stop the water from flowing through the faucet assembly.  I had to go under the house and turn off the main water line.

Next, I pried off the "H" and "C" cover with a screwdriver.  This showed a screw, which I removed.  The handle could then be removed.



I then used a wrench to take off the bonnet.

NOTE:  Don't use too much force here.  You don't want to damage the plumbing behind the faucet.  If it's stuck, don't pull with all your might.  Try loosening it up with a hammer.

This exposed the retainer ring, which holds in the first washer and the ball.   Both can then be pulled out by hand.   You can then easily access the washer and replace.   

If you look in the remaining hole, you'll see the valve seats.  I used a pair of needle-nose pliers to get those suckers out, along with their corresponding springs.  I put the new valve seats on the springs and placed on the pliers for reinsertion like so:  

I then stuck both back into the hole.  

After that, everything was put back in its place.  On reassembly, be sure the notch on the packing retainer ring goes into the slot correctly on the wall.  Also, don't put the retainer ring too tightly.  It should be fine with just a hand tightening.  If it's too tight, the valve seats will get worn out faster and the faucet will be difficult to use.

The main water line was then turned back on and the faucet was tested.  

No more leak!

Updated 2/23/2014 with a couple of small notes.

Repairing a Small Retaining Wall

My 35 year old house has reached the age where unexpected things are starting to wear out.  For instance, I had to replace my bathtub drain lever a month or so ago.

One of the many other items starting to show its age is the retaining wall separating my front yard from the road.  It's very small, only about a foot high, but it's starting to become unsightly, particularly the part near my mailbox.  The main reason:  people continue to run into it while backing out of the driveway.

The wall is made out of railroad ties two pieces high. The bottom railroad tie is okay, but the top railroad tie is starting to decompose.  I need to replace it.

I couldn't find much on the internet about this job, but it seemed pretty straightforward. The most challenging part of this was actually finding somebody to sell me a railroad tie.  None of the big box hardware stores (Lowes, Home Depot, Ace, etc) have them in stock.  I finally found somebody in Raleigh/ Cary, Family Home and Garden, that sold them (only $15 each!).  Note:  Railroad ties are pretty heavy.   Although Family Home and Garden will help you get them in your automobile/ trailer, you'll probably want somebody to help you unload them. 

Because people loved driving their cars into the railroad tie near the mailbox, I also wanted to make sure it stayed put when pushed with a lot of force.  I found these three foot solid metal spikes for about $4 each next to the rebar (my initial choice) at Home Depot.

These should be easier to contend with than rebar due to their pointy ends.  I bought four of them.  

In regards to tools, an engineering hammer (2 lbs), wheelbarrow, reciprocating saw (with a decent blade - I used the ones rated for demolition), drill (powerful one with some pretty sizable bits), transfer shovel, trowel, and crowbar were all employed for this task.  YMMV, however, depending on your circumstances.

First up, removing the current, unsightly railroad tie. I live in the woods, so I beat on the old railroad tie for a minute or two to give any current tenants (mice, snakes, etc) an eviction notice.  I next used a crowbar to pry off the old railroad tie.  It was fastened previously with long nails, which I also removed from the top and bottom railroad tie.  

I next prepared the spot for the new railroad tie.  I used a transfer shovel to scrape off all the dirt, rotten wood and other yard gunk.  Using a trowel, I also removed some of the earth being retained behind the wall.  The purpose was so I could set the railroad tie back a couple of extra inches and prevent it from jutting out.  

Then, the new railroad tie tried out its new home.  Make sure you like the way it looks in its spot.  It might not hurt to try flipping it over and comparing other positions.  Mine was a little warped and looked better when rotated.  I also took a little more dirt out  so I could wedge it in better and make it more flush with the retained dirt wall. 

After getting everything situated, it's time to drill.  This took forever since my drill is pretty old and weak. Here's a picture of it.  It was a hand-me-down from my father.

I used a 1" bit and slowly made a hole through the middle of one of the ends of the tie (about a foot from the edge).  Be careful.  My piece was pretty dense and had a few rocks in it.  

I then drove my spike through the drilled hole and the bottom railroad tie with a two pound engineering hammer.

After that, I repeated the drilling and hammering process three more times, once on the other end and twice more closer to the middle.  The new railroad tie is in place!

What about the old rotting one?  I cut it up with a reciprocating saw and hauled it off to the dump.

Overall, this was a pretty easy project.  I probably invested a couple of hours completing it (not counting the time spent hunting down the replacement railroad tie).

Here's the end product.  You can get an idea of the condition of the other railroad ties by comparing the end of the older one next to the new one.  I'll replace the others as time permits.

Stuff I Like - Coursera

I recently transferred into a new and exciting data analytics department within my employer.  When they chose me to be part of the team, I was thrilled, but a little anxious.  My background isn't really in analytics.

It was time to roll up the sleeves and get up to speed on this analytics field that I was going to join.  But, where do I start?!?!

Well, I found this great website called Coursera.  Coursera provides a place for universities to have online courses on just about everything.  The schools are pretty prestigious too.  UNC-Chapel Hill, Duke, and a large number of other schools all are represented.

So far, I've taken  (or am currently taking) a class on data analytics, statistics and data science, but there are classes on just about everything.  The professors post online videos, assign homework, and even give you a grade at the end.  They have forums where students can ask teachers (and other students) questions.  So far, it's been the best way I've found to start acclimating myself to this new profession.

And the best part?  It's completely free!  Well, not counting the work you'll have to put in to get the knowledge in your head...

Resetting your HDHomeRun TV Tuner Automatically

Time Warner Cable is expensive.  Despite that, I still fork over money to them every month.  I try to reduce my costs as much as I can by only subscribing to only their most basic packages, such as the just the basic cable package, which provides only the network channels (ABC, NBC, CBS, FOX, etc) for around $20 a month.

We don't have one of their cable boxes, which can allow you to record channels and watch them later.  A wonderful service, but I'm way too cheap for that.  Instead, I use a computer to record the channels through the wonderful  SageTV software.  Unfortunately, the SageTV company was bought out by Google and the software can no longer be purchased.

To tune in the channels with the computer, the SageTV software interacts with a nifty device from Silicon Dust called the HDHomerun.  It's a TV tuner that hooks up to your network.  I've been using one of these for years with little complaint.  

Note, though, that I said "little." I had an older model that kept misbehaving.  It took me a couple of months of troubleshooting before giving up and just replacing the device.  Before I got there, I tried a couple of other approaches.  I thought my computer might be the culprit, so I rebuilt it and set it up to reboot every night (as I discussed here a couple of weeks ago).   I also set up the HDHomerun TV tuner to reboot nightly through a quick and dirty program and a scheduled task.

Creating the means to reset the HDHomerun was actually a lot easier than I thought.  I first found this document from Silicon Dust explaining how you can interact with a Silicon Dust program installed on my computer that interacts with the tune (hdhomerun_config.exe).  I then found the following function in the application available through their help file.

set /sys/restart self

This is the command I need to restart the HDHomerun tuner.

I created a bat file with the following line through Notepad.

"c:\Program Files\Silicondust\HDHomeRun\hdhomerun_config.exe"  10324a85 set /sys/restart self

The 10324a85  is the unique number given to my tuner so the hdhomerun_config program knows which tuner to reset.  You can get this number through the HDHomerun Setup program which is accessible through the Start menu. The unique number is listed right here:

I then set up a scheduled task to run this program regularly to reset my tuner.  

For instructions on creating the bat file and setting up the scheduled tasks, I discussed both in a post I did a couple of weeks ago on how to set up your PC to reboot regularly.  The process should be pretty similar.

Even though I've replaced the tuner I was originally using when I set this up, I still reset the new tuner every night.  I figure it doesn't hurt anything.

I'm happy with my setup, but wish I could continue to reduce my cable bill.  In a month or so, I'm going to start experimenting with antennas to see if I can get TV reception over the air.  I'm a bit skeptical, though.  My house is in a pretty wooded area and I'm not too fond of mounting an antenna on my roof.  

Will my frugality and curiosity win out over my fear of heights?  Stay tuned!