Colour Palette Creator

I am sure that there are hundreds of tools like this on the web, but I thought it would be quite interesting to produce a colour palette based on an image. Imaging you are designing a website and you really like the colours used in a particular piece of art, then you've come to the right place. This software will extract the most common colours in an image, and also give you the average colour.

The method used to calculate these is very simple. To calculate the average colour, simply sum all the RGB components of each pixel and divide by the total number of pixels. I do this using bitwise operators where:

R = HEX >> 16 & 0xFF
G = HEX >> 8 & 0xFF
B = HEX & 0xFF

and HEX = (R<<16 | G<<8 | B), useful stuff!

To calculate the palette I create an accumulator array which counts the number of each colour in the image. Before using this I reduce the number of colours in the image using some simple math.

I hope you find it quite interesting to have a play around with, just click the image to launch it. You can load your own images from your local computer to play around with it.

Computer Vision: Regional Differencing

I have always been very excited by the idea of computer vision. At the same time I find the concept of teaching a computer to sift through a grid of pixels, whilst gaining some sort of an insight into their meaning quite astounding. When we as humans look at a scene we can immediately identify objects of interest to us. We look at a boat and immediately can tell that it is a boat and not for example a chair. We can do this, even if we have never seen that particular boat before. Computers struggle with this kind of visual finesse, although there are examples (see Assimo) that are already performing very well.

Back to the point: I was looking at this situation and thought to myself that humans find it incredibly easy to establish important features in any image. Take for example the image to the left, we can see clearly the line where the water meets land at the far edge of the lake, we can see trees, we can see a ridge and its reflection, and we can see clouds. These geometries are made visible to use because of contrast and colour differences between regions in the image. I decided to think of a way for a computer to do this. I'm sure the method has been done before under a different name, but I couldn't find any material on it so I've decided to call the method "Regional Differencing".

The method is fairly simple, but I am happy with the results. First the image is split into square regions of a constant size. The colour of these regions is equal to the average colour of the pixels contained within it. The result of this is a kind of pixilation as can be seen in the picture to the right. Now taking the colour values of every single pixel in the previous image, we can compare the value to that of the average regional colour.

The result of this is quite interesting because areas that don't fit in with their surroundings are clearly distinguishable in the image. In this article I'd like to argue that this is part of the reason why humans are so good at picking out the geometries mentioned above. Humans can happily distinguish between different regions, even where differences are relatively subtle. Combining this ability with memory of collections of shapes, we can identify objects very successfully.

So lets take a look at some of the results (as always click the first one to launch the application and try it out yourself):

The above 3 images show three different threshold levels for a constant pixel size, but if you run the app yourself you can also vary the pixel size yielding interesting results. At high thresholds it is interesting that the first objects that are made out are the trees and the line at the end of the lake, whereas the last objects made out are the clouds. Looking back I wonder if I found myself looking at these points before anything else.

There are many other ways to produced these kinds of data, including the Sobel operator method shown in various previous posts but sometimes it is nice to take a new approach.

Regional Growth Segmentation: Superpixels

I recently came across some impressive software from some researchers at Stanford University (who else), called Make3D which takes any 2 dimensional images - preferably outdoor images, and converts them into 3D models. The whole 2D to 3D process begins with a method called image segmentation, where structures in an image are separated into superpixels. Superpixels are collections of pixels with similar properties - for example saturation, colour, brightness etc.

I decided to write some software capable of creating the superpixel structure, and segmenting an image. The route I went down is called regional growth segmentation. Superpixels grow outwards from a sample pixel, and depending on the similarity of tested pixels to the superpixels, they are either added to the superpixel, or create new superpixels...

Here are a few results with different thresholds for similarity (again, just click the first image to launch the application).

I'll take a look at some of the Make3D source code to see if I can generate some 3D planes from my superpixel, so stay tuned!

Image Evolution

Based on the work by Roger Alsing I thought I'd create a real time evolving image. Initially 30 random numbers are generated from a seed, these numbers are the components of the DNA strand. Each number corresponds to a specific polygon with a characteristic colour, transparency and number of vertices. A mutation in the DNA strand results in a shape changes. Images before and after a mutation are compared to the original painting, Girl With A Pearl Earring, and if a mutation causes greater similarity, then the mutation holds, otherwise the DNA reverts to its previous form. This is similar to how evolution works in the biological world.

Here are the first few mutations showing development of the image:

These are early stages, with up to 165 successful mutations. I will leave the program running over night, and will update the results. The lack of computational power in AS3 and my completely un-optimised code means that comparison processes take quite a while. One way to improve speed would be to reduce canvas size so I may turn to this if the program doesn't produce anything nice.

Having said that the early results are making me feel optimistic. We just need a few more thousand mutations and it'll be there.

Update: Here are the 198th and 238th mutations, I don't think the image will look that interesting until it gets into the 1000s. Its getting there but its clear that its a long process. Unfortunately the rate of improvement also slows down. Its up to about 15000 mutation attempts, and its taking longer and longer to mutate successfully each time. Currently only up to 10 vertices are allowed per polygon. Increasing this maximum number to more could increase the rate of improvement. As could allowing other sorts of mutation such as gene swapping, where two of the seed numbers swap. I'll give this a go soon although I will let this version go to 1000 mutations just to see the results.



Update: Just a though on using this as a compression technique. Each seed number is a 32bit integer. This means 30 numbers comes to 960bits or 120bytes per image. Given that the original version of the image I used (a PNG so already compressed) comes to 231kBytes, thats a compression factor of about 2000. Even using 100 polygons (a factor which would drastically improve the final evolution of the image) the compression factor would still be around 600! This is very impressive. I remember reading about a competition somewhere to send images as twitter messages (a twitter message contains up to 120 characters or 120bytes of ascii?) This means that the images I have produced could indeed be sent as twitter messages.




Update: The evolution process seems to be levelling off at about 360 successful evolutions with over 1.6 Million evolution attempts. This leads me to believe that my evolution parameters weren't ideal. The two images below shows the image as far as I'm going to take it. The second shows a blurred image applied and actually the result is pretty good when compared to the blurred original (in other words when you squint you should be able to see the original image. I'll come back when to you when I've improved the parameters and algorithm slightly!

Face Recognition Algorithms

Over the last few days I have been looking at a various algorithms and methods used in face recognition processes. The first method I looked at uses a colour space known as TSL (closely related to HSL - hue, saturation, luminosity). This colour space was developed with the intention of being far closer to the way the human brain sees colours than RGB and other computationally preferred systems. Taking a webcam stream and converting all pixels into TSL colour space with [R,G,B] -> [T,S,L], I found that although in certain lighting conditions the system can differentiate between skin colour and other surfaces, the method is far from ideal. For example the cream walls in my house can often be identified as skin colour, which clearly shows an issue. It is also clear from the images that areas of my face that are in shade are not recognised as skin. These issues are best addressed using methods that do not depend on colour.











In my research I found two viable methods for use in face recognition, using eigenfaces, and fisherfaces.

Eigenfaces seemed to be a more commonly used method, so I decided to follow that route first (albeit roughly, as I'm sure you all know by now I like to do things my own way when I program). In order to recognise a face in an image, the computer has to be trained to know what a face looks like. The first step involved writing a class to import .pgm files from the CMU face database. Here is a sample of what the faces looked like when imported. All images are in the frontal pose,  with similar lighting conditions, and varying facial expressions.

The next three steps involved creating an average face, and taking the resulting image and calculating it's vertical and horizontal gradients. The average face simply sums all of the pixels at each location for each face, and takes the average value. To calculate the gradient the difference between two adjacent pixels is taken in either the vertical or the horizontal direction. Computers find it easy to see vertical and horizontal lines (I have already written some basic shape detection software which uses these kinds of algorithms) so I thought this might be a good idea to use these as comparisons with found faces. I planned on using a kind of probability test, with a threshold as to the likeliness that any part of the image is a face, by comparing it to the mean face, the horizontal gradient face, and the vertical gradient face.

The three faces found are shown below for this database. Clearly one could find the mean face of all people wearing glasses, or all men, or all women, and this would affect it's final appearance. Therefore it could theoretically be simple to build in gender testing using webcams (assuming a complete lack of androgyny which clearly there is not....), but a probabilistic approach could still be taken.

The mean face looks incredibly symmetric and smooth. This is perfection folks, and its kind of frightening! The idea behind using these for face recognition is relatively simple. Scan an image taking the difference between an overlaid mean face, and the region of the image being scanned. If the difference is below a threshold it means that the images are similar. This means it is likely that there is a face where you are checking. To ensure it is a face consider the horizontal and vertical gradients of the mean and compare them. If they are similar to within a certain threshold it is very likely you have found a face!

I'll come back to you when I have some working flash files and source code!

Ported Mandelbrot Explorer for Flash AS3

Hi everyone - I've got some very very exciting news!

I've made a flash version of the project I've been working on at uni with export functionality.

Click here to launch it

The features list is as follows:

Colour Settings
 -> Red, Green, Blue
 -> Greyscale
 -> 3 types of multicoloured rendering

Form Settings
 -> Classic Mandelbrot
 -> Conjugate Mandelbrot
 -> Absolute Mandelbrot
 -> Imaginary Absolute Mandelbrot
 -> Real Mandelbrot

Dimension Settings
 -> 2,3,4,5,6

Preview Image
 -> 150px by 120px

Output Image
 -> png format
 -> 400-2000 width, 300-2000 height

Panning and Zoom - outputted location to interface.

Rendering is never more than about 20seconds even for 2000x2000 pixels on my 2GHz 2GB iMac which is pretty good considering its flash!

Here is a little render of a Burning Ship Fractal at (-1.88480, -0.00041) with a zoom of 126988x. Just click to view it full size!

Save functionality on CS4!

Finally what we've all been waiting for and it's really simple to implement.

I recently had to write some software for Braindash, a company I've been working for on and off for the last few years, who deal in puzzle games. The software was a brain training IQ puzzle generator thing, which had to take a puzzle and batch export several images for each question (a question and 4 answers) as well as a csv data file with information about the questions.

I actually didn't have a clue that writing data with flash without any serverside scripting was possible, so I'd given up on the idea after doing some reasearch into it a year or so ago. But now thats all changed which is absolutely fantastic.

File saving is done using the FileReference class, and the save method.

Implementation is as easy as:

var fileToSave:FileReference = new FileReference();
fileToSave.save(data,"name"+".extension");

where data can be:

- A String
- XML
- ByteArray data

If data is none of these, toString(); is called on whatever variables are in the data.

In the case of my work I used the AS3corelib PNGencoder to handle creating the byteArray from bitmapData in flash.

Enjoy this class, try making software which creates random art every time you open it and saves it to your desktop, pretty cool methinks!

Cheers

Sam