We built this computer vision system that can play Rockband as our final project for CS184 at UC Berkeley.

In the space of two weekends we designed and built a system that uses computer vision to monitor the xbox display through a camera and play the game. More details can be found at njoubert.com/assets/cs184/rockband/Rockband.pdf

We were, well, sleep deprived for a good section of this work, which explains the craziness in some parts of the video.

Our system is similar to Slashbot and AutoGuitarHero, but we do not take a video feed from the console - no, we're doing it through a panasonic handycam pointed at the screen. We're interested in machine vision, and this was a fun project to get into the field!