Hurricane Irma was forcast to hit Miami and work it’s way up the whole state. A lot of people decided to leave and there wasn’t really any place to go except north. I live in Gainesville, right on one of the two north-south interstates in the state - I75. Our city runs traffic cameras and the make current images available on a SmartTraffic site. A few months ago I wrote a script to pull all the images once a minute so I have a history of all the “current” images.

I thought it would be fun to stich them together to make a time-lapse video of the interstate. I started it around Sept 1 so you can see normal traffic, the evacuation rush starts around 9/6, and it runs up until the traffic system crashed at 1:45 am on Monday morning which is about 15 minutes after the peak winds were forcast to hit.

Video on Youtube:

Some details about how this was made:

• The I75 cameras are movable with 4 pre-programmed positions. They take one picture per minute so there’s one image about every 4 minutes. You’ll also see some times when the camera didn’t properly return to it’s pre-programmed position.
• The orientation of the camera isn’t saved in the image metadata (who would have thought that would be useful!) but it is displayed in the image. I OCR’ed the orientation from a crop in the image with pyocr and tesseract. It was pretty good but you’ll see some stray images from other positions where the OCR messed up. I’m not a tesseract wizard.
• There are gaps in the images for a number of reasons. Sometimes the camera goes out. Sometimes the images are partial downloads since I’m pulling a jpg off a web server every minute and they’re uploading to the same file name every minute. I get about 5% corrupt or partial files.
• The final image is just the last image uploaded off the camera before it died. It just happened to be in the right orientation so for the last 2 days I was pulling the same image of their web site. Its all a hack.

So far I have download ~40 cameras worth of data since February 2017 every minute for a total of about 750 GB and 16 million images. Not sure what else I’ll do with them.

And here it is almost a year since my last post but I now have something worthwhile again. Below is my application essay for the Industrial and Systems Engineering department’s Patterson award at UF. I felt it was a good summary of what I’ve done over the course of my studies at the University of Florida on my way to finishing my Masters’s in Industrial Engineering.

As a non-traditional student, the Industrial Engineering Master’s program at the University of Florida has been a transformative experience. I took classes part time for three years and the longer time scale provided me with an opportunity to relate each class in turn to my career. It allowed time for my connection to the UF community to become much stronger by providing more opportunity to interact with multiple cohorts of students across many semesters. My degree, and the experiences it represents, have given me new opportunities in my current job and will allow me to direct my career towards my goal of doing open data science research with the biodiversity community in the future.

The Industrial Engineering master’s degree program at UF offers flexibility. After I found a new interest in data science during the operations research course, I was able to take related courses in the Computer and Information Science and Engineering department. The courses in Algorithmic Economics and Bioinformatics provided specific applications of the mathematics and statistics from the core Industrial Engineering courses. During my final semester I completed a Special Projects class supervised by Dr. Panos Pardalos. I presented my project titled “Towards Mining Whole Museum Collections Datasets for Expanding Understanding of Natural History Collections” to an audience of industrial engineering graduate students and biodiversity professionals. I truly enjoy bringing together diverse groups and helping them find where their interests overlap.

The most significant contribution I made to the UF community was helping to found the Data Science and Informatics student organization (DSI). My role in DSI has been a blend of both student-learner and advisor. My knowledge and enthusiasm towards data science was similar to the other student members and we learned as peers during workshops and joint projects. I have also been able to bring my computing, managerial, and institutional knowledge to the group in an advisory capacity. While the student officers picked the direction of the group, I provided them with techniques for hosting great events, technical infrastructure, and faculty participants for their events. Along with everyone else’s enthusiasm and hard work, my contributions have been critical to building a wildly successful data science community which has grown to more than 500 members in just one year.

After graduating this spring, I will be continuing in my current position at the Advanced Computing and Information Systems Lab in the Electrical and Computer Engineering department. After discussions with the lab’s director, I am now responsible for producing collaborations that tie the research done within our lab to research done in other domains. One of the first examples of this will be extending my final project to include collaborators from the biodiversity informatics community. We are expecting to present a platform that allows biologists to use big data analytics tools like Apache Spark on museum collections data at conferences in Berlin and Costa Rica later this year.

Here is documentation of me presenting my final project titled “Towards Mining Whole Museum Collections Datasets for Expanding Understanding of Natural History Collections”

The University of Vermont’s 1993-94 undergraduate catalogue lists CS 011 Computer Programming I as a first semester course for freshman in the civil engineering bachelor’s program. I however entered UVM in the fall of 1993 as an environmental studies major so I did not take it. When I switched to civil engineering the next year, the recommended programming course had changed to CS 016 Programming in MATLAB and as Frost said, that has made all the difference.

That course was a new addition taught for the first time by Maggie Eppstein. She was a lecturer in 1994 but is now the UVM computer science department chair. This was my first experiance programming and it was a revelation. I fell in love with the act of creating something that worked just the way I wanted using a langage that was completely logical. Every programming concept Maggie introduced just seemed obvious and natural. I don’t remember ever studying or reviewing anything after lectures, I just did the programming homework in a couple hours and aced all the exams.

I got an A+ in the class (nerd!) and enjoyed it so much that I signed up to be a TA the next semster. For 5 to 10 hours a week I sat in office hours working through homeworks with struggling students and ran around in the hands-on labs trying to keep everyone caught up with Maggie who would be scrawling code on the whiteboard. I kept TA’ing that course for 5 semesters until I graduated. It was the first job that I chose to persue myself - no parents involved.

I have been thinking about my experiance for two reasons: first because I read this article in the Spring 2015 UVM College of Engineering and Mathmatical Sciences Summit newsletter. The protagonist, Caitlyn Bishop, is a former biology major at UVM who took the same course nearly 20 years later as an elective and also fell in love with programming. She ended up switching to math and computer science as a result of her acidental discovery. I was supprised by how coincidently similar her discovery of programming was and I started wondering how often this happens.

The second is that I just went through the Software Carpentry instructor training program. In our final conference call, Greg Wilson said something along the lines of “Congratulations on completing your training, now let’s get you signed up to teach some workshops.” This was the moment where I started to wonder “Wait, what did I just volunteer for?”

The answer to that is that I volunteered to help people decide if programming is for them. That is really what I enjoyed about TA’ing MATLAB and that is what I want to have an opportunity to do again.

And maybe like Caitlyn, some SwC students can find out early what they are truely interested in to make the most their time in college or early careers. I haven’t designed a beam in 20 years and I’m still catching up on basic math and computer science concepts because I didn’t follow the programming road on to where I should have: computer science.

Some more fun college computing stories:

When came for my orientation, the engineering school made a big deal about how they had just finished upgrading the 386’s in the main computer lab with DX co-processors so they could use the new 3-D drafting appliations.

Sophomore year was the first year UVM offered wired network connections in dorm rooms. But they blocked IPX networking which DOOM used for multi-player so we bought thin-net coax network cards (no expensive hub required!) and ran cables down the halls to network our computers together.

The day I got my computer in my dorm room hooked up to the Internet, one of my friends cam over and started showing me gopher and FTP sites. Some time during that session I started wondering if I could some how limit my paper topics to just subjects that I could find references for on FTP sites so I wouldn’t have to go to the library ever again. I don’t think I ever checked out a book in my 4 years at UVM.

At the start of what must have been the second mid-term in Maggie’s MATLAB class, she walked in with a pillow case half filled with Halloween candy. Apparently she had some sort of disciplinary issue with her kids and they had their candy confiscated. She brought it in for us since she couldn’t eat it all. To this day I always feel just a touch let down when I walk in to an exam and there is no candy.