Part 2 – Creating YouTube live streams programmatically
Authentication with the YouTube Data API from Python
This is the second project for which I’ve had to go through and figure this out, so I wanted to write this down. There are a couple oddities with the YouTube Data API:
One great thing about computers is that they can be programmed to do things that are repetitive and boring. I try to look around in my life to see what things I can get a computer to do for me, so that I don’t have to do it myself.
Today’s case is scheduling weekly YouTube live streams for my church. Every week someone’s got to schedule the live streams for the upcoming weekend. They look like this:
So you might be thinking, this seems pretty trivial, like it’s just a few clicks to schedule this in YouTube. It can’t take more than 10 minutes. And that’s true, but it’s still straightforward and repetitive. Having to figure out which Sunday is coming up, copying and pasting and ensuring the correct dates and times are replaced in the live stream text, and making sure the scheduled dates and times are correct can become tedious work. And it’s the same procedure week after week: the type of processing that computers love to do.
Sure there are other ways to optimize this process, like batching it to create maybe two or more weeks at a time. However, from a viewer’s perspective it can also be confusing when there are a bunch of upcoming live streams that need to be scrolled through. For the purposes of this project, the optimal frequency is to have one set of upcoming live streams visible at any time, and that means having the computer schedule the next set of live streams once every week.
So let’s get the computer to do this. I’m going to split up this blog post into two parts:
I haven’t had time to work on many side projects lately. I think I generally am most effective at programming when I have a long stretch of uninterrupted time (on the order of a few hours or more at a time), and I have found the same at work. These long stretches of time are difficult to find these days.
However this was one opportunity that came up this week. Many of my side projects are born from automating repetitive manual tasks, and this was one of those.
I’ve been running my “T-Comm” Bus Locator website for over seven years now. As a quick recap, on the backend, it’s a procedural PHP/MySQL website, kind of a typical thing one would see of PHP development circa 2000s.
The high level overview is that every minute or two, the system polls TransLink’s real time information APIs for the locations of the all the buses in the system. The system then calculates some information based on TransLink’s GTFS data, which has the schedule information for the entire system. Then it saves the data into a MySQL database, and some other outputs for use on the web interface.
So naturally as more buses operate during the daytime (up to around a thousand buses), the CPU load increases during the day, then drops off at night (down to several on the Night Bus routes). It could take about two minutes to calculate all the necessary information to update all the buses during the daytime
CPU load over the last week
Last night, I set off to find a quick way to optimize this, without having to rewrite the entire system (which arguably it sort of needs, but that’s for another day). From a cursory at the system resources using tools such as htop, it was pretty obvious that the bottleneck was on the MySQL database and its reading and writing to the relatively slow hard disks.
At work, we have been receiving reports of people, particularly on Android 2, not being able to install from the Google Play Store citing an error message “Package file is invalid“. At the same time, I was trying to test one of the features I was developing on a couple of Android 2 test devices we have in the office, but I was repeatedly getting an INSTALL_FAILED_DEXOPT error from adb. It seemed that both of these issues could be related. Continue Reading
