Key detection for a virtual piano teacher

We propose a method for identifying a piano keyboard present in the video footage of a standard webcam with the goal of teaching chords, scales and suggested finger positions to a beginner pianist. Our keyboard identification method makes use of binary thresholding, Sobel operators and Hough transforms, as well as proposed algorithms specific to this application, to first find an area resembling a piano keyboard before narrowing the search to detect individual keys. Through the use of our method the keys of a piano keyboard were successfully identified from webcam video footage, with a tolerance to camera movement and occluded keys demonstrated. This result allowed the augmented reality style highlighting of individual keys, and the display of suggested fingering, for various chords and scales - which demonstrates the potential for our piano teacher program as a learning tool. The demo application achieved an average frame rate of 25.1 frames per second when run on a 2.20GHz dual-core laptop with 4GB RAM; a suitable rate for real-time use.