Voice Recognition in Flash

This evening I thought I'd take a crack at looking at voice recognition in flash. The idea for this came after the realisation that when using my phone I have almost entirely given up on using the keypad. Instead I simply utter a few words (When passers by are out of ear shot) and a voice appears at the other end. Wonderful! I haven't read up on voice recognition at all so in part I am using the knowledge I've gained by looking at image recognition and tracking software. Being a big Aphex Twin fan and having written a spectrograph to visualize this song in flash before I had at least a bit of a grounding. The idea would be to record a small sample of voice, using some microphone thresholding, draw a spectograph of the sample and then compare the spectograph to a database of possible patterns and determine which command or whatever is most similar. I'm pleased to say that I have completed the first half of this today and you can click to see a demo below (The top image will launch it):

The images are of a whistling sequence, a glass being rung, and me saying hello. Interestingly you can see the harmonics of the glass being rung (a few set frequencies). My voice is a bit harder to translate but using some analysis I'm sure it will be possible.

Anyway as usual the source code can be downloaded here and let me know if you come up with anything using it. How about auto-tuning?!

The plan from here is to save single voice clips as images, and train a neural network to find patterns in both the volume and the spectograph for a set number of commands. I will report my findings shortly!

Perceptron in Actionscript

I've always thought neural networks and the idea of computational learning was pretty cool but until now I'd never actually tried to program one. I decided to start out with something simple so I went for the single layer perceptron with the hope of building it up to something a bit smarter.

For those of you who don't know much about neural networks Wikipedia has done a pretty decent job in teaching me everything I know. All the project files are available here and sample source code is provided below. It is a simple solution to the NAND problem using two inputs.

The perceptron class essentially has two functions, train and evaluate. The train function is called 100 times on the training set which adjusts the respective weights on each input. When evaluate is called on a particular inputs an output is produced. The output trace from the below program looks like this:
Initiating new perceptron:

Training 100 times using training set...

Trained

Evaluating NAND[0,0] : 1

Evaluating NAND[1,0] : 1

Evaluating NAND[0,1] : 1

Evaluating NAND[1,1] : 0

This is a fast solution to these kinds of problems. I plan on implementing optimisations such as exiting training when the weights change by less than some threshold (i.e the solution limit has been reached), but the end goal is to produce a full library with many kinds of next step. I think the next of these will be a multilayer perceptron which initial research suggests is much more complicated! Enjoy the classes and let me know if you do anything interesting with them.

package

{

import flash.display.Sprite;

import org.experimentalized.networks.SingleLayerPerceptron;

import org.experimentalized.networks.objects.Trainer;

import org.experimentalized.networks.objects.TrainerSet;

public class Main extends Sprite

{

public function Main()

{

//NAND example

trace("Initiating new perceptron:");

var perceptron:SingleLayerPerceptron = new SingleLayerPerceptron(0.1);

var training_set:TrainerSet = new TrainerSet([ new Trainer([0,0],1),

new Trainer([1,0],1),

new Trainer([0,1],1),

new Trainer([1,1],0)

]);

trace("Training 100 times using training set...");

perceptron.TrainSet(training_set,100);

trace("Trained");

//Test some evaluations

trace("Evaluating NAND[0,0] :",perceptron.Evaluate([1,0]));

trace("Evaluating NAND[1,0] :",perceptron.Evaluate([1,0]));

trace("Evaluating NAND[0,1] :",perceptron.Evaluate([0,1]));

trace("Evaluating NAND[1,1] :",perceptron.Evaluate([1,1]));

}

}

}

Colour Wheel - HSV Colour Space

I've just been playing around with colour wheels and colour representation in flash. I had a think about whether it was possible to represent every single colour in the RGB spectrum on a single 2D image. Closer inspection reveals that it isn't possible because RGB is made up of three components - red, green and blue - and therefore needs three dimensions. This is why if you look up different colour representations on Wikipedia you might see them represented by cones, or cubes. I decided to work in the HSV colour space which is  formed of Hue (rotation), Saturation (The amount of colour compared to black/white) and Value (Essentially the brightness). The resulting colour wheel can be seen below:

Just click on the image to launch the app. By rolling over the wheel the colour in the box will show the selected colour. Use the up/down arrow keys to alter the "V" value of the wheel. This should allow you to view a complete (discrete) representation of every colour in the RGB spectrum. 

If you're interested in source code just click here to download the FlashBuilder project and AS files. Enjoy!

There's nothing boring about Tax!

Well the title might be a bit misleading, but I'll let you be the judge of that. A close friend and I were in the pub last week discussing the benefits of a tax system which, instead of being tiered, would be characterised by a continuous curve. My friend being an Economist, and me a Physicist, and both of us slightly tipsy thought that we might be able to change the world with our new theory - so I went home that night and wrote a small program using FlashMath to try and quantify some of our ideas. The program essentially integrated over a Cauchy-Schwartz curve and compared collected tax with that of the standard tiered system. I had planned on expanding on this by optimising the curve using measures such as utility (an economic measure of happiness of a population) but for now here is the source code:

package org.experimentalized
{
 package
{
 import flash.display.MovieClip;
 import com.experimentalized.math.quadrature.SimpsonsQuadrature;

 public class Tax extends MovieClip
 {
  private var gamma:Number = 7000;
  private var x0:Number = 22000;
  private var population:Number = 65000000;
  
  private var gamma2:Number = 12000;
  private var x1:Number = 30000;
  private var beta:Number = 0.52;
  
  public function Tax()
  {   
   trace(calculate_total_teired_tax_collected(1000000));
   trace(calculate_total_new_tax_collected(1000000));
  }
  
  public function calculate_total_new_tax_collected(Limit:Number):Number
  {
   var S:SimpsonsQuadrature = new SimpsonsQuadrature(new_tax_proportion,0,Limit,100);
   return S.integrate();
  }
  
  public function new_tax_proportion(X:Number):Number
  {
   return population*calculate_new_tax_collected(X)*evaluate_income_percentile(X);
  }
  
  public function calculate_new_tax_collected(income:Number):Number
  {
   var S:SimpsonsQuadrature = new SimpsonsQuadrature(lorenzian_cdf,0,income+0.001,100);
   return S.integrate();
  }
  
  public function lorenzian_cdf(X:Number):Number
  {
   return (1/Math.PI)*Math.atan((X-x1)/gamma2)*beta + beta/2;
  }
  
  public function calculate_total_teired_tax_collected(Limit:Number):Number
  {
   var S:SimpsonsQuadrature = new SimpsonsQuadrature(teired_tax_proportion,0,Limit,100);
   return S.integrate();
  }
   
  public function teired_tax_proportion(X:Number):Number
  {
   return population*calculate_teired_tax_collected(X)*evaluate_income_percentile(X);
  }
  
  public function calculate_teired_tax_collected(income:Number):Number
  {
   var taxable:Number = income - 7475;
   if(income>100000) taxable = income;
   
   var basic_rate:Number = taxable;
   if(basic_rate>35000) basic_rate = 35000;
   
   taxable = taxable - basic_rate;
   
   var higher_rate:Number = taxable;
   if(higher_rate>115000) higher_rate = 115000;
   
   taxable = taxable - higher_rate;
   
   var additional_rate:Number = taxable;   
   
   if(basic_rate<0) basic_rate = 0;
   if(higher_rate<0) higher_rate = 0;
   if(additional_rate<0) additional_rate = 0;
   
   return basic_rate * 0.2 + higher_rate * 0.4 + additional_rate * 0.5;
  }
    
  public function evaluate_income_percentile(X:Number):Number
  {
   //Model as lorenzian
   return (1/Math.PI)*(gamma/((X-x0)*(X-x0) + gamma*gamma))   
  }
 }
}

It isn't commented in the slightest but see what you can make out of it!

Fun with Inverse Kinematics

In part to try out AXDT, a flash plugin for Eclipse (which you can find here), I decided to create a simple experiment involving Inverse Kinematics. This is not a very useful implementation but the results are quite fun. Essentially, the angles of subsections of  each kinematic chain are altered until they face the mouse. I quickly discovered this is not the best solution for IK, but just for fun I made a kind of windswept, fur or grass effect using about 200 IK chains. 

As usual click the top screenshot to launch the experiment and click here for source:

Update: Colourful Turtle

In my last experiment I recreated a turtle vector drawing program, with a simple interface. I'll post a small update to that which adds some color to the output. Here are some screenshots, as usual click the first one to launch the image, and source code can be found at the bottom, enjoy and let me know if you make anything pretty!

Source:

package org.experimentalized
{
 import flash.display.Sprite;
 import flash.events.Event;
 import flash.events.KeyboardEvent;
 import flash.display.Graphics;
 import flash.text.TextField;
 import flash.text.TextFieldType;
 import flash.geom.Rectangle;
 import flash.utils.getTimer;

 [SWF(width='550',height='550',backgroundColor='#000000',frameRate='30')]

 public class Turtle extends Sprite
 {
  private var commandstring:String = new String("mv:30,lt:90,mv:10,rt:30");
  private var S:Sprite = new Sprite();
  private var Angle:Number = new Number(0);

  private var X:Number = 0;
  private var Y:Number = 0;

  private var R:int = 0;
  private var G:int = 0;
  private var B:int = 0;


  private var input:TextField = new TextField();

  
  private function newrun(e:KeyboardEvent):void
  {
    commandstring = input.text;
    S.graphics.clear();
    S.graphics.lineStyle(1,0xFFFFFF);
    S.graphics.moveTo(0,0);
    Angle = 0;
    X = 0;
    Y = 0;
  }

  public function Turtle():void
  {
   addEventListener(KeyboardEvent.KEY_DOWN, newrun);
  
   addChild(S);

   input.width = 530;
   input.height = 20;
                 input.x = 10;
                 input.y = 10;
                 input.background = true;
                 input.backgroundColor = 0xFFFFFF;
   input.type = TextFieldType.INPUT;
   
   input.text = commandstring;


   addChild(input);

   S.x = 275;
   S.y = 200;
      
   this.addEventListener(Event.ENTER_FRAME, handler);
  }
  
  private function handler(e:Event):void
  {
   R = int(128 * (1 + Math.cos(getTimer()/1000-0.8981)));
   G = int(128 * (1 + Math.sin(getTimer()/7473-0.3812)));
   B = int(128 * (1 + Math.sin(getTimer()/1273+0.1787)));
   
   var C:uint = (R<<16|G<<8|B);   
   
   S.graphics.lineStyle(1,C);
   var commands:Array = new Array();
   commandstring = commandstring.split(" ").join("");
   commands = commandstring.split(",");
   var i:int;

   for(i=0;i<commands.length;i++)
   {
    if(commands[i].charAt(0)=="m")
    {
     var dist:Number = Number(commands[i].split(" ").join("").split(":")[1]);
     X = X + dist*Math.cos(Angle);
     Y = Y + dist*Math.sin(Angle);
     S.graphics.lineTo(X,Y);
    }
    else if(commands[i].charAt(0)=="l")
    {
     var changeleft:Number = Number(commands[i].split(" ").join("").split(":")[1]);
     Angle -= Math.PI*changeleft/180;
     if(Angle<0) Angle += 2*Math.PI;
    }
    else if(commands[i].charAt(0)=="r")
    {
     var changeright:Number = Number(commands[i].split(" ").join("").split(":")[1]);
     Angle += Math.PI*changeright/180;
     if(Angle>Math.PI*2) Angle -= 2*Math.PI;
    }
   }
  

   //Resize to fit on the square after each iteration (purely aesthetic choice...)
   
   var Rect:Rectangle = S.getRect(S.parent);
   
   S.x += 275 - (Rect.x+Rect.width/2);
   S.y += 275 - (Rect.y+Rect.height/2);
   S.width = 550;
   S.height = 550;
  }
 }
}

Turtle: 30 minute experiment

I had recently been reminiscing about IT lessons in year 3 where we were allowed to play around with those Turtle vector drawing programs. I thought I'd write one and share the results as a quick break from revision. Although it can't do much at the moment (the only commands are forward, left and right) with a bit more work I'm sure it could be made into quite an exciting program. You'll find the source code below and as usual just click the top image to launch the experiment.

Source code:

package org.experimentalized
{
 import flash.display.Sprite;
 import flash.events.Event;
 import flash.events.KeyboardEvent;
 import flash.display.BitmapData;
 import flash.display.Bitmap;
 import flash.display.Graphics;
 import flash.text.TextField;
 import flash.text.TextFieldType;

 [SWF(width='550',height='400',backgroundColor='#000000',frameRate='30')]

 public class Turtle extends Sprite
 {
  

  private var commandstring:String = new String("mv:30,lt:90,mv:10,rt:30");
  private var S:Sprite = new Sprite();
  private var Angle:Number = new Number(0);

  private var X:Number = 0;
  private var Y:Number = 0;

  private var input:TextField = new TextField();

  
  private function newrun(e:KeyboardEvent):void
  {
    commandstring = input.text;
    S.graphics.clear();
    S.graphics.lineStyle(1,0xFFFFFF);
    S.graphics.moveTo(0,0);
    Angle = 0;
    X = 0;
    Y = 0;
  }

  public function Turtle():void
  {
   addEventListener(KeyboardEvent.KEY_DOWN, newrun);
  
   addChild(S);

   input.width = 530;
   input.height = 20;
                 input.x = 10;
                 input.y = 10;
                 input.background = true;
                 input.backgroundColor = 0xFFFFFF;
   input.type = TextFieldType.INPUT;
   
   input.text = commandstring;


   addChild(input);

   S.x = 275;
   S.y = 200;
   
   S.graphics.lineStyle(1,0xFFFFFF); 
   
   this.addEventListener(Event.ENTER_FRAME, handler);
  }
  
  private function handler(e:Event):void
  {
   var commands:Array = new Array();
   commandstring = commandstring.split(" ").join("");
   commands = commandstring.split(",");
   var i:int;

   for(i=0;i<commands.length;i++)
   {
    if(commands[i].charAt(0)=="m")
    {
     var dist:Number = Number(commands[i].split(" ").join("").split(":")[1]);
     X = X + dist*Math.cos(Angle);
     Y = Y + dist*Math.sin(Angle);
     S.graphics.lineTo(X,Y);
    }
    else if(commands[i].charAt(0)=="l")
    {
     var changeleft:Number = Number(commands[i].split(" ").join("").split(":")[1]);
     Angle -= Math.PI*changeleft/180;
     if(Angle<0) Angle += 2*Math.PI;
    }
    else if(commands[i].charAt(0)=="r")
    {
     var changeright:Number = Number(commands[i].split(" ").join("").split(":")[1]);
     Angle += Math.PI*changeright/180;
     if(Angle>Math.PI*2) Angle -= 2*Math.PI;
    }
   } 
  }
 }
}

To make the output lines automatically fit the stage use the following instead:

package org.experimentalized
{
 import flash.display.Sprite;
 import flash.events.Event;
 import flash.events.KeyboardEvent;
 import flash.display.Graphics;
 import flash.text.TextField;
 import flash.text.TextFieldType;
 import flash.geom.Rectangle;

 [SWF(width='550',height='550',backgroundColor='#000000',frameRate='30')]

 public class Turtle extends Sprite
 {
  private var commandstring:String = new String("mv:30,lt:90,mv:10,rt:30");
  private var S:Sprite = new Sprite();
  private var Angle:Number = new Number(0);

  private var X:Number = 0;
  private var Y:Number = 0;

  private var input:TextField = new TextField();
  
  private function newrun(e:KeyboardEvent):void
  {
    commandstring = input.text;
    S.graphics.clear();
    S.graphics.lineStyle(1,0xFFFFFF);
    S.graphics.moveTo(0,0);
    Angle = 0;
    X = 0;
    Y = 0;
  }

  public function Turtle():void
  {
   addEventListener(KeyboardEvent.KEY_DOWN, newrun);
  
   addChild(S);

   input.width = 530;
   input.height = 20;
                 input.x = 10;
                 input.y = 10;
                 input.background = true;
                 input.backgroundColor = 0xFFFFFF;
   input.type = TextFieldType.INPUT;
   
   input.text = commandstring;


   addChild(input);

   S.x = 275;
   S.y = 200;
      
   this.addEventListener(Event.ENTER_FRAME, handler);
  }
  
  private function handler(e:Event):void
  {
   
   S.graphics.lineStyle(1,0xFFFFFF);
   var commands:Array = new Array();
   commandstring = commandstring.split(" ").join("");
   commands = commandstring.split(",");
   var i:int;

   for(i=0;i<commands.length;i++)
   {
    if(commands[i].charAt(0)=="m")
    {
     var dist:Number = Number(commands[i].split(" ").join("").split(":")[1]);
     X = X + dist*Math.cos(Angle);
     Y = Y + dist*Math.sin(Angle);
     S.graphics.lineTo(X,Y);
    }
    else if(commands[i].charAt(0)=="l")
    {
     var changeleft:Number = Number(commands[i].split(" ").join("").split(":")[1]);
     Angle -= Math.PI*changeleft/180;
     if(Angle<0) Angle += 2*Math.PI;
    }
    else if(commands[i].charAt(0)=="r")
    {
     var changeright:Number = Number(commands[i].split(" ").join("").split(":")[1]);
     Angle += Math.PI*changeright/180;
     if(Angle>Math.PI*2) Angle -= 2*Math.PI;
    }
   }
  

   //Resize to fit on the square after each iteration (purely aesthetic choice...)
   
   var Rect:Rectangle = S.getRect(S.parent);
   
   S.x += 275 - (Rect.x+Rect.width/2);
   S.y += 275 - (Rect.y+Rect.height/2);
   S.width = 550;
   S.height = 550;
  }
 }
}

ASTrace Pre Release

Although the library is nowhere near complete, I thought I would release a beta to see what you think. At this stage feedback would be really useful, including any suggestions you might have about extensions to the project. I should note that there is no documentation as of yet, this will be coming as soon as I manage to get asdoc to run properly. There should be plenty of scope for you to play around with the library, it'd be great if you could send me some of your renders.

Here is a link to the library: ASTRACE

and here are some of my favourite renders using the library: