Grabbing all of the links to a Youtube Playlist in Python

My friend Jesse wanted a way to grab all of the links to videos on a YouTube playlist. That’s no problem. I can easily cycle through links and spit them out into an HTML page.

There’s just one bug that I can’t seem to fix. Each and every link in the output goes to Rick Astley - Never Gonna Give You Up. I guess some bugs were never meant to be solved.

Examining Exit Codes of 4 simple programs with bugs: Python, Ruby, Perl, PHP

Strict type checking is an aspect of the compiler to make sure that your programs have properly declared all of the necessary variable names prior to their use. I believe it is essential to all modern languages to implement proper errors in the event that an undefined variable is used. Let’s look at 4 popular scripting languages to see how they handle this problem.

For each program, we will examine three things: the source code, the execution, and the returned exit code. In modern operating systems, a program that concludes execution successfully returns a code of 0. If any problems arise in the course of the program, it should return something other than 0. I used the UNIX command “echo $?” to check the exit code.

All 4 programs will print the word “Hi” to the screen, then it will try to print an undefined variable “thisDoesNotExist”. After that, it will print “Hi again”. We would hope that the program will abort some time during the execution of the program and tell us that “thisDoesNotExist” is undefined. We would also hope that the program would never return an exit code of 0.

First, my favorite little language: Python

badName.py:

print "Hi"
print thisDoesNotExist
print "Hi again"

Execution and return code:

jcchurch@mccarthy:~/tmp/exitCodes$ python badName.py
Hi
Traceback (most recent call last):
  File "badName.py", line 2, in <module>
    print thisDoesNotExist
NameError: name 'thisDoesNotExist' is not defined
jcchurch@mccarthy:~/tmp/exitCodes$ echo $?
1

Python gives me a good error message “NameError: name ‘thisDoesNotExist’ is not defined”. It also returns a status code of 1. This is what we should expect from a modern language.

Second: Ruby

badName.rb:

puts "Hi"
puts thisDoesNotExist
puts "Hi again"

Execution and return code:

jcchurch@mccarthy:~/tmp/exitCodes$ ruby badName.rb
Hi
badName.rb:2: undefined local variable or method `thisDoesNotExist' for main:Object (NameError)
jcchurch@mccarthy:~/tmp/exitCodes$ echo $?
1

Ruby also gives a good error message “undefined local variable or method `thisDoesNotExist’” and an exit code of 1.

Third: Perl

Not every language designer agrees with me that strict type checking is essential. Perl requires us to use the statement “use strict;” to use strict type checking. I wish this were turned on by default.

badName.pl:

use strict;

print "Hi\n";
print $thisDoesNotExist;
print "Hi again\n";

Execution and return code:

jcchurch@mccarthy:~/tmp/exitCodes$ perl badName.pl
Global symbol "$thisDoesNotExist" requires explicit package name at badName.pl line 4.
Execution of badName.pl aborted due to compilation errors.
jcchurch@mccarthy:~/tmp/exitCodes$ echo $?
255

Perl doesn’t even print “Hi”. It gives the error message “Global symbol “$thisDoesNotExist” requires explicit package name”, which is a tad cryptic, but it does the job we want it to do. It also returns with a exit code of 255.

Fourth: PHP

I’m breaking my rule of never posting PHP code on this site. Like Perl, strict type checking is turned off by default. We can turn it on with the statement “error_reporting(E_ALL);”.

badName.php:

<?
error_reporting(E_ALL);
print "Hi\n";
print $thisDoesNotExist;
print "Hi again\n";
?>

Execution and return code:

jcchurch@mccarthy:~/tmp/exitCodes$ php badName.php 
Hi

Notice: Undefined variable: thisDoesNotExist in /home/jcchurch/tmp/exitCodes/badName.php on line 4
Hi again
jcchurch@mccarthy:~/tmp/exitCodes$ echo $?
0

PHP gives us a good error message: “Undefined variable: thisDoesNotExist”. But wait! It keeps going! We see “Hi again” AFTER the error message, which means this isn’t really an error message but a warning. When the program concludes, the exit code returned to the operating system is 0, which tells me that nothing went wrong in the program.

Conclusion

Python and Ruby demonstrate the same behavior: Execute as much of the program as possible until an error is reached. Once the error is reached, print an error and abort the program.

Perl compiles the entire program and notices the undefined variable at compile time. It prints the error without executing any of the code.

PHP executes as much of the code as possible. When it encounters the undefined variable, it happily prints a warning message and keeps on going. Finally, it tells us that there were no problems in the code with an exit code of 0.

Oh PHP. Maybe you will grow up as a language one of these days.

Splitting Pages of a PDF in Python

For the past month, I’ve programmed in nothing but Matlab. Kinda sad, because I don’t even care for Matlab. I prefer Python.

I was going through a store called Tuesday Morning to look at the stuff for sale. It’s a bit of a junk store. They had a DVD for sale containing every Fantastic Four comic book on PDF from 1961 to 2004 for only $15. Being a comic fan, I had to get it.

Each issue is scanned into PDF format. Each 2 pages of the paper issue is combined into 1 page of the PDF. I didn’t like this, so I decided to write a Python script to cut each page of the PDF down the middle into two pages and stitch the document back together. This means that 1 page of a comic book equals 1 page of PDF.

I use is the handy pyPdf library. It’s doing all of the magic in this script. To execute this script:

$ python splitPages.py InputDocument.pdf OutputDocument.pdf

(Since the comics are intellectual property owned by Marvel, I’m going to not post screen shots.)

from pyPdf import PdfFileWriter, PdfFileReader
import sys

print "Reading", sys.argv[1]

output = PdfFileWriter()

left = PdfFileReader(file(sys.argv[1], "rb"))
right = PdfFileReader(file(sys.argv[1], "rb"))

left.decrypt('')
right.decrypt('')

pages = left.getNumPages()

for i in range(0, pages):
# Grab the left page
p = left.getPage(i)
p.mediaBox.upperRight = (
p.mediaBox.getUpperRight_x() / 2,
p.mediaBox.getUpperRight_y()
)
output.addPage(p)

# Grab the right page
p = right.getPage(i)
p.mediaBox.upperLeft = (
p.mediaBox.getUpperRight_x() / 2,
p.mediaBox.getUpperRight_y()
)
output.addPage(p)

print "Writing", sys.argv[2]

outputStream = file(sys.argv[2], "wb")
output.write(outputStream)
outputStream.close()
Downloading Stock Market Data via Yahoo using Python
"""

In my spare time, I'm an investor. I figure I should
should use my powers in math and computer science and
help build for myself a better future.

This program will download the past year's stock
history for a list of ticker symbols defined by the
'symbols' list variable. Each stock's data will be
downloaded to your computer into a CSV file, which
can easily be viewed with Microsoft Excel.

Tomorrow's post will be a program on how to read the
output files using the popular statistical computing
software R.

"""

import urllib
from datetime import date

symbols = [ "BAC", "YHOO", "RIMM", "MU", "MSFT", "GOOG", "PFE", "INTC", "C", "GE", "SAP", "YUM", "CLRT", "ORCL", "AMZN", "XTXI", "GS", "KO", "F", "PG", "EMR", "AAPL", "KFT", "NFLX", "CPNO", "BIG", "FTR"]

def download(url, outfile):
    webFile = urllib.urlopen(url)
    localFile = open(outfile, 'w')
    localFile.write(webFile.read())
    webFile.close()
    localFile.close()

def get1YearPriceHistory(symbol):
    today = date.today()
    url = "http://ichart.finance.yahoo.com/table.csv?s=%s&a=%02d&b=%02d&c=%04d&d=%02d&e=%02d&f=%04d&g=d&ignore=.csv" % \
           (symbol, today.month - 1, today.day, today.year - 1, today.month - 1, today.day, today.year)
    outfile = symbol + ".csv"
    download(url, outfile)

if __name__ == '__main__':
    for s in symbols:
        get1YearPriceHistory(s)
Simple Animations in Python

My Frog Pond

from graphics import *
import time

"""

In the CS 141 class that I teach at Rhodes College we
use a simple graphics library written by Dr. John Zelle
to demonstrate basic object oriented programming topics.

My assignment to the class was to use the graphics library
to tell a simple story. This program tells the story
of a frog sitting on a lily pad made of gold coins (I'm
not sure why I put in the Chinese cultural reference) who
is hungry, so he hopes that a fly comes.

A fly does come, the frog eats the fly, and the frog is happy.


Dr. Zelle's graphics library can be downloaded from here:
http://mcsp.wartburg.edu/zelle/python/

Note: The class thought I went overboard with this
demonstration.

"""

def drawCoin(win, x, y):
    coin = Oval(Point(x, y), Point(x+25, y+10))
    coin.setFill('yellow')
    coin.draw(win)

    blacksquare = Rectangle(Point(x+11, y+4), Point(x+13, y+6))
    blacksquare.setFill('black')
    blacksquare.draw(win)

def drawFrog(win, x, y):

    # This is the Left Foot
    p1 = Point(x+25, y+85)
    p2 = Point(x+10, y+100)
    p3 = Point(x+40, y+100)
    foot1 = Polygon(p1, p2, p3)
    foot1.setFill('green')
    foot1.draw(win)

    # Right Foot
    p4 = Point(x+75, y+85)
    p5 = Point(x+60, y+100)
    p6 = Point(x+90, y+100)
    foot2 = Polygon(p4, p5, p6)
    foot2.setFill('green')
    foot2.draw(win)

    frog = Oval(Point(x,y), Point(x+100, y+100))
    frog.setFill('green')
    frog.draw(win)    

    # Left Eye
    eye1 = Oval(Point(x, y), Point(x+25, y+25))
    eye1.setFill('white')
    eye1.draw(win)

    # Right Eye
    eye2 = Oval(Point(x+75, y), Point(x+100, y+25))
    eye2.setFill('white')
    eye2.draw(win)

    # Left Pupil
    pupil1 = Circle(Point(x+12, y+12), 4)
    pupil1.setFill('black')
    pupil1.draw(win)

    # Right Pupil
    pupil2 = Circle(Point(x+75+12, y+12), 4)
    pupil2.setFill('black')
    pupil2.draw(win)

    # The mouth
    line1 = Line(Point(x+12, y+50), Point(x+25, y+60)); line1.draw(win)
    line2 = Line(Point(x+25, y+60), Point(x+75, y+60)); line2.draw(win)
    line3 = Line(Point(x+75, y+60), Point(x+87, y+50)); line3.draw(win)

def drawFly(win, x, y):
    flyBody = Rectangle(Point(x,y), Point(x+20, y+4))
    flyBody.setFill('white')
    flyBody.draw(win)

    flyWing = Oval(Point(x+4,y-20), Point(x+8, y))
    flyWing.setFill('white')
    flyWing.draw(win)

    for i in range(3):
        # Move up and to the left
        for j in range(50):
            time.sleep(0.05)
            flyBody.move(-1,-1)
            flyWing.move(-1,-1)
        # Move down and to the left
        for j in range(50):
            time.sleep(0.05)
            flyBody.move(-1,1)
            flyWing.move(-1,1)

    return (flyBody, flyWing)

def drawTongue(win, x, y, flyBody, flyWing):
    p1 = flyBody.getP1()
    p2 = Point(p1.getX()+5, p1.getY())
    p3 = Point(x+70, y+60)
    p4 = Point(x+75, y+60)
    tongue = Polygon(p1,p2,p3,p4)
    tongue.setFill('red')
    tongue.draw(win)

    time.sleep(1)

    tongue.undraw()
    flyBody.undraw()
    flyWing.undraw()
    time.sleep(2.0)

def drawPond(win):
    sky = Rectangle(Point(0,0), Point(399, 249)) 
    sky.setFill( color_rgb(0, 0, 255) )
    sky.draw(win)

    water = Rectangle(Point(0, 250), Point(399, 399))
    water.setFill( color_rgb(0, 0, 128) )
    water.draw(win)

    # Draw 20 gold coins for the frog's lily pad
    y = 5
    while y > 0: 
        x = 0
        while x < 4:
            drawCoin(win, 25 + 25 * x, 300 + 10 * y)
            x += 1
        y -= 1

    # And the frog...
    drawFrog(win, 25, 200)

    speechBubble = Oval(Point(175, 200), Point(375, 300))
    speechBubble.setFill('white')
    speechBubble.draw(win)

    message = Text(Point(275, 250), "Hi")
    message.draw(win)
    time.sleep(2.0)

    message.setText("I'm a frog."); time.sleep(2.0)
    message.setText("Could you tell?"); time.sleep(2.0)
    message.setText("My lily pad is gold coins!"); time.sleep(2.0)
    message.setText("Anyway, I'm also a lazy frog."); time.sleep(2.0)
    message.setText("I hope a fly comes by."); time.sleep(2.0)
    message.setText("Here comes one now."); time.sleep(2.0)

    message.undraw()
    speechBubble.undraw()

    (flyBody, flyWing) = drawFly(win, 400, 100)

    # Now to animate the tounge!
    drawTongue(win, 25, 200, flyBody, flyWing)

    speechBubble.draw(win)
    message.draw(win)
    message.setText("Yum! Good!")

def main():
    win = GraphWin("My Frog Pond", 400, 400)
    drawPond(win)
    win.getMouse()
    win.close()

main()
Filtering a BibTeX file by terms used in the Keywords tag in Python
import sys
import re

"""
Author: James Church
Date: November 16, 2010
Filename: filterByKeyword.py

This program will gather up all of the citations in a BibTeX
file that share a keyword passed to it on the command line.

For example, say you have a list of citations in a file
"docs.bib", but some docs are tagged with the keyword
"ants" and other are tagged with the keyword "beetles".
Let's get a list of all of the citations involving ants:

python filterByKeyword.py docs.bib ants
"""

def main():
    cite = ""
    keyword = sys.argv[2]
    for line in open(sys.argv[1]):
        if line[0] == '@':
            tag = re.search("{\s*(\w+)", line)
            cite = tag.group(1)
        if keyword in line and "keywords" in line:
            print "\cite{"+cite+"}"

if __name__ == '__main__':
    sys.exit(main())
The Coin Change Problem in Python
def coin_change(amount, denominations, collection=[]):
    """
    Given an amount of money and some coin denominations, this
    function prints every combination of coins needed to express
    the desired amount using the input denominations.

    amount - an amount, in the smallest unit.
    denominations - a list of coin denominations, from largest to smallest.
    """
    for coin in denominations:
        if coin == amount:
           print collection+[coin]
        if coin < amount:
           coin_change(amount-coin, denominations, collection+[coin])

coin_change(10,[25,10,5,1])