Grabbing all of the links to a Youtube Playlist in Python

My friend Jesse wanted a way to grab all of the links to videos on a YouTube playlist. That’s no problem. I can easily cycle through links and spit them out into an HTML page.

There’s just one bug that I can’t seem to fix. Each and every link in the output goes to Rick Astley - Never Gonna Give You Up. I guess some bugs were never meant to be solved.

Mandelbrot Set in Matlab

I wanted to see if I could create a Mandelbrot Fractal image in Matlab. After quickly looking at the Mandelbrot Set entry on Wikipedia, I coded up the following and ran it:

I’m kinda surprised that it took so little code to create the image. I’m pleased with the results.

Finding a 3D Rotation Matrix for two vectors in Matlab

Update: The formulation of the input has been updated to allow two matrices of 3 by n column vectors and return a 3 by 3 by n matrix of rotation matrices.

Here’s the problem:

I have a 3 by 1 vector and I need to know the rotation matrix to manipulate that vector to another 3 by 1 vector.

Let’s call the first vector ‘u’ and the second vector ‘v’. I’m looking for a 3 by 3 rotation matrix ‘R’ that satisfies this simple property:

R*u=v

That’s it!

I’m sure that you’ve encourtered solving systems of linear equations in an algebra class. They are pretty painful, right? In those cases, you are given ‘R’ and ‘v’ and you are told to solve for vector ‘u’. By comparison, those are easy. Here we go with the solution:

Solving Many, Many 3 by 3 Eigenvalue Problems in Matlab

My research is requiring me to solve many, many small eigenvalue problems, all of which are 3 by 3 matrices. Rather than use the built-in “eig” function, I wrote my own.

Determining the Circumcircles of a list of Triangles in Matlab.

Imagine that we have a set of two dimensional points scattered across a space. For each point, the X coordinates are in variable X, and the Y coordinates are in variable Y. We can passed these coordiantes into the “delaunay” function and get “TRI”, which is a list of triangles. We can view the triangle mesh of the Delaunay triangulation by passing it to the MATLAB function “trimesh”, but there is no function for viewing the circumcirlces produced by this function. That’s what I’ve created today.

This second block isn’t my code. I found it on the Mathworks website. I’m posting it here to demonstrate that the above code works. It’s just used for plotting the circles.

Let’s make some triangle meshes!

This produces the elegant picture below.

Examining Exit Codes of 4 simple programs with bugs: Python, Ruby, Perl, PHP

Strict type checking is an aspect of the compiler to make sure that your programs have properly declared all of the necessary variable names prior to their use. I believe it is essential to all modern languages to implement proper errors in the event that an undefined variable is used. Let’s look at 4 popular scripting languages to see how they handle this problem.

For each program, we will examine three things: the source code, the execution, and the returned exit code. In modern operating systems, a program that concludes execution successfully returns a code of 0. If any problems arise in the course of the program, it should return something other than 0. I used the UNIX command “echo $?” to check the exit code.

All 4 programs will print the word “Hi” to the screen, then it will try to print an undefined variable “thisDoesNotExist”. After that, it will print “Hi again”. We would hope that the program will abort some time during the execution of the program and tell us that “thisDoesNotExist” is undefined. We would also hope that the program would never return an exit code of 0.

First, my favorite little language: Python

badName.py:

print "Hi"
print thisDoesNotExist
print "Hi again"

Execution and return code:

jcchurch@mccarthy:~/tmp/exitCodes$ python badName.py
Hi
Traceback (most recent call last):
  File "badName.py", line 2, in <module>
    print thisDoesNotExist
NameError: name 'thisDoesNotExist' is not defined
jcchurch@mccarthy:~/tmp/exitCodes$ echo $?
1

Python gives me a good error message “NameError: name ‘thisDoesNotExist’ is not defined”. It also returns a status code of 1. This is what we should expect from a modern language.

Second: Ruby

badName.rb:

puts "Hi"
puts thisDoesNotExist
puts "Hi again"

Execution and return code:

jcchurch@mccarthy:~/tmp/exitCodes$ ruby badName.rb
Hi
badName.rb:2: undefined local variable or method `thisDoesNotExist' for main:Object (NameError)
jcchurch@mccarthy:~/tmp/exitCodes$ echo $?
1

Ruby also gives a good error message “undefined local variable or method `thisDoesNotExist’” and an exit code of 1.

Third: Perl

Not every language designer agrees with me that strict type checking is essential. Perl requires us to use the statement “use strict;” to use strict type checking. I wish this were turned on by default.

badName.pl:

use strict;

print "Hi\n";
print $thisDoesNotExist;
print "Hi again\n";

Execution and return code:

jcchurch@mccarthy:~/tmp/exitCodes$ perl badName.pl
Global symbol "$thisDoesNotExist" requires explicit package name at badName.pl line 4.
Execution of badName.pl aborted due to compilation errors.
jcchurch@mccarthy:~/tmp/exitCodes$ echo $?
255

Perl doesn’t even print “Hi”. It gives the error message “Global symbol “$thisDoesNotExist” requires explicit package name”, which is a tad cryptic, but it does the job we want it to do. It also returns with a exit code of 255.

Fourth: PHP

I’m breaking my rule of never posting PHP code on this site. Like Perl, strict type checking is turned off by default. We can turn it on with the statement “error_reporting(E_ALL);”.

badName.php:

<?
error_reporting(E_ALL);
print "Hi\n";
print $thisDoesNotExist;
print "Hi again\n";
?>

Execution and return code:

jcchurch@mccarthy:~/tmp/exitCodes$ php badName.php 
Hi

Notice: Undefined variable: thisDoesNotExist in /home/jcchurch/tmp/exitCodes/badName.php on line 4
Hi again
jcchurch@mccarthy:~/tmp/exitCodes$ echo $?
0

PHP gives us a good error message: “Undefined variable: thisDoesNotExist”. But wait! It keeps going! We see “Hi again” AFTER the error message, which means this isn’t really an error message but a warning. When the program concludes, the exit code returned to the operating system is 0, which tells me that nothing went wrong in the program.

Conclusion

Python and Ruby demonstrate the same behavior: Execute as much of the program as possible until an error is reached. Once the error is reached, print an error and abort the program.

Perl compiles the entire program and notices the undefined variable at compile time. It prints the error without executing any of the code.

PHP executes as much of the code as possible. When it encounters the undefined variable, it happily prints a warning message and keeps on going. Finally, it tells us that there were no problems in the code with an exit code of 0.

Oh PHP. Maybe you will grow up as a language one of these days.

Top 10 List for Categorical Data in MySQL

Let’s say that you have a table with some categorical data. You want to make a Top 10 list of the most frequently visited categories. Here is an example table:

In this case, we have a table called “example” with one column called “label”. This is a bad example, in that it only has 10 items in the whole table.


mysql> SELECT * FROM example;
+-------+
| label |
+-------+
| A |
| B |
| C |
| D |
| A |
| B |
| C |
| A |
| B |
| A |
| E |
| E |
| E |
| E |
| E |
+-------+
15 rows in set (0.00 sec)

We wish to create a Top 10 List (err… Top 5 List) of the most frequently seen categories (or labels) in our table. Here is the code:


mysql> SELECT label, n FROM (SELECT label, count(*) AS n FROM example GROUP BY label) AS T1 ORDER BY T1.n DESC LIMIT 10;
+-------+---+
| label | n |
+-------+---+
| E | 5 |
| A | 4 |
| B | 3 |
| C | 2 |
| D | 1 |
+-------+---+
5 rows in set (0.00 sec)

There are two SELECT statements here! Let’s look at the inner SELECT statement. Here it is:


mysql> SELECT label, count(*) AS n FROM example GROUP BY label;
+-------+---+
| label | n |
+-------+---+
| A | 4 |
| B | 3 |
| C | 2 |
| D | 1 |
| E | 5 |
+-------+---+
5 rows in set (0.00 sec)

This bit of code groups labels together, pulls the label column and the count of each label (and names this count as ‘n’). It almost does the job, except the data isn’t sorted. You might wonder why we don’t use “ORDER BY n DESC” here. At this point, we’ve called a new column ‘n’, but we cannot use that new column in filtering or ordering.

Let’s move to the outer SELECT statement. I’ve replace the inner SELECT statement with “…”.


mysql> SELECT label, n FROM (...) AS T1 ORDER BY T1.n DESC LIMIT 10;
+-------+---+
| label | n |
+-------+---+
| E | 5 |
| A | 4 |
| B | 3 |
| C | 2 |
| D | 1 |
+-------+---+
5 rows in set (0.01 sec)

We are using the inner SELECT statement as a brand new temporary table for the outer SELECT statement. You can see the “AS T1”, which means that we are naming our inner SELECT statement “T1”. We order by T1’s column ‘n’ in descending order, limit the output to 10 records and again pull the columns ‘label’ and ‘n’.

In summary, creating a sorted ‘Top 10 List’ in MySQL requires a nested SELECT statement.

Determining if a graph is connected in Matlab

I needed a quick script to tell me if a graph was connected or not. My adviser usually tells me “MATLAB has everything!” No, it doesn’t, and that’s why we need to write simple code like this just to get more complicated jobs done.

Since I haven’t posted in a while, let’s do another script. This one will determine if a graph is symmetrical, which is another simple problem that MATLAB doesn’t have the code to do. As you can see, the code is very short.