Voice Recognition in Flash

This evening I thought I'd take a crack at looking at voice recognition in flash. The idea for this came after the realisation that when using my phone I have almost entirely given up on using the keypad. Instead I simply utter a few words (When passers by are out of ear shot) and a voice appears at the other end. Wonderful! I haven't read up on voice recognition at all so in part I am using the knowledge I've gained by looking at image recognition and tracking software. Being a big Aphex Twin fan and having written a spectrograph to visualize this song in flash before I had at least a bit of a grounding. The idea would be to record a small sample of voice, using some microphone thresholding, draw a spectograph of the sample and then compare the spectograph to a database of possible patterns and determine which command or whatever is most similar. I'm pleased to say that I have completed the first half of this today and you can click to see a demo below (The top image will launch it):

The images are of a whistling sequence, a glass being rung, and me saying hello. Interestingly you can see the harmonics of the glass being rung (a few set frequencies). My voice is a bit harder to translate but using some analysis I'm sure it will be possible.

Anyway as usual the source code can be downloaded here and let me know if you come up with anything using it. How about auto-tuning?!

The plan from here is to save single voice clips as images, and train a neural network to find patterns in both the volume and the spectograph for a set number of commands. I will report my findings shortly!

Simple 3D Music Visualizer

Just played around a bit with the 3D fluid dynamics thing I made a few days ago. I added some computeSpectrum functionality using the Flash 10 audio features and created a very basic music visualizer. I'm not entirely happy with it but its worth a post, just click to start it up (the song might take a while to load):

The song is "Rabbit In Your Headlights" by Unkle, enjoy!

iTunes in Adobe Air

I use itunes a lot and it struck me to think how apple decided to store all of my playlist information, song name, album, artist, composer, track length and so on... So I opened up my iTunes Music Library.xml file and saw what I can only describe as a complete mess. Apple have made it very difficult indeed to find what you are looking for in their bundle of seemingly random XML nodes. Below is an example of the information stored for a single song.

A complete and utter mess if you ask me. Anyway I went about foolishly trying to decode the alien structures. A word to apple, why not just label your XML tags with relevant names, like track ID, instead of calling every single one ? So after a few hours I had a fully working iTunes emulator, running perfectly well with 20000 or so tracks all loaded into a dataGrid component. Due to the shear amount of XML (mostly wasted) songs couldn't all be loaded at once which meant a small wait until the Air application had parsed every line. I think that if apple ordered it's XML in a slightly more coherent and sensible fashion then the task would have been far easier. Anyway here is a screenshot of what I came up with.

I also added the music visualizer to the application and now I have my very own lightweight itunes, get in touch if you're interested in the source for either of these things.

AS3 Music Visualizer using flashes new spectrum analyzer

Turn your speakers up for this one! I've been playing around with the bitmap class and particle effects a lot recently and have been coming up with some interesting often unexpected effects. This is a perfect example of trying to do one thing and creating something completely different. Initially I was trying to create a flocking model using bitmap data. Along the lines of the 1980s computer program boids. I wanted the particles to follow the position of the mouse, but after a slightly flawed line of code, they ran away from the mouse instead. Having played with the spectrumAnalyze function a few days previously I thought it would be interesting to make the particles "run" away from mappings of the spectrum values, instead of the mouse. With 30,000+ particles and a blur filter I made the following. (Sorry if it takes a while to load! - click the image below to load it...)

The potential in flash to create on the fly music visualizers is now bigger than ever. I have seen some fantastic ones floating around on the internet, I just wish the spectrumAnalyze function worked with microphone inputs!!! Maybe next release eh?

here is a slightly different version of the visualizer, see what you think... (For some reason the music doesn't work with this one, don't know why that is.....

Enjoy.