Webcam Gestures in Flash

I'm currently creating a game which involves gesturing as the primary control method. For those who aren't familiar with the term gesturing, it was created to describe a set of motions carried out on a touch screen. I thought it would be pretty cool to make the keyboard and mouse completely obsolete and applied my gesturing algorithm to the webcam. Getting timings right with the webcam was the hardest part. You can choose when to start and stop using a mouse, but how do you tell when one gesture ends and the next begins on the webcam.

Click below to watch a video of the experiment in action:

It works by calculating the angles travelled in a gesture. It compares these to a list of angles which relates to a certain gesture. For example for a straight line in the positive horizontal direction this would be 0-0-0-0-0 or in the negative horizontal 180-180-180-180. The program determines which preset gesture the carried out gesture is most like and then performs an action based on this choice. The result is by no means stable, but using a better set of thresholds and presets can improve the performance dramatically.

Click here to launch the experiment and have a go yourself. Remember you'll need a webcam to do so.