Saturday, January 14, 2006

motion capture in python

You know those suits with the ping-pong balls attached to it? The kind they use in motion capture? I thought it would be really cool if I could build my own. Then I'd be able to film my kickboxing and analyze it afterwards.

The current state-of-the-art seems to be tracking a magnetic field with the sensors on the suit. Unfortunately I don't have the resources to do that. So I'm going with the older way: visually tracking the markers.

I capture frames using my cheap webcam and the VideoCapture Python library. Others have used it for simple motion detection but not motion tracking. I decided to track the motion of red objects. Why red? Well, it's one of the least common colours in my office and is usually concentrated to a small area (bottle caps, mugs, books).

To get my hands on the red pixels I can't simply extract the R from RGB images. If a colour has a red component it could also be white, yellow, magenta... So you take the R and subtract the G and B. Ignoring values below zero you now have a gauge for the "redness" of a pixel.

(Extra points: Correct me if I'm wrong, but I believe "R minus G minus B" will give you a "redness" plane that intersects the xy chromaticity diagram (see sRGB color space) as a line running from blue to green along the edge of the sRGB gamut. The triangular plane segment within the gamut reaches it's "peak" at the maximum red representable by sRGB.)

This is a simple way to find the maximum red point on the image. I drew a crosshair over that point for clarity. With the series of images captured by the webcam I constructed an animated gif. So the system tracks a red object. Cool.

However, what I want to do is track more than one red object. I did this by forming clusters of red dots above certain intensity.

How do you form clusters? Well I used the K-means algorithm, conveniently included with SciPy. Et viola! It seems to do a pretty good job of it.

Not bad for a days work. Only 150 lines of Python. Of course it is at this point that my fear is confirmed: My webcam doesn't seem up to the task of tracking the speed of my movements. So before I jump around with a bunch of red markers on my body, I'll need a better camera.

Edit: You can download a rough version of the code here: Parts are commented out so you don't need a webcam. You will need Python, SciPy and PIL however. I hope this works...

Another Edit: Here's an alternative I wrote ( that uses numarray, Pycluster and PIL instead. I'm starting to dislike SciPy. If there are enough requests I think I'll have to do another write-up.


Anonymous said...

that looks pretty cool, sooooo where's the code? :)

Janto Dreijer said...

Your wish is my command :)

Anonymous said...

Oh man, we don't get to see the kickboxing videos?

Janto Dreijer said...

The project was placed on hold. I need to improve on the update speed. So: no, you don't get to see any videos :(

Anonymous said...

I really like this code.
I have a bunch of ideas for programs that could incorporate this code.
the only problem is, I can no longer get it to work on my computer. I installed all the extra stuff (scipy, PIL, etc) but I get this error:
File "", line 12, in ?
from scipy.cluster import vq
ImportError: No module named cluster

I was wondering you have any insight to this.

Janto Dreijer said...

I uploaded an alternative (see post) that removes the SciPy dependancy. You'll have to use Pycluster, however.

I'd love to see what you can do with this basic idea.

Anonymous said...

I don't know what is up, but I still can't get it to work. but that is fine.
you should check out my motion track script. It is a basic idea, let me know what you think. (phillipmates[at]
example video

phillip. said...

my post above has broken links. python webcam color tracking (with a link to this post)

Anonymous said...

i instaled and it says i need module named scipy....where can i download it and after i download it do i need anything elese for it to run??? my e-mail is
send me a mail please...
sory for bad english :)))

Janto Dreijer said...

You can download scipy from I don't think you need anything else for it to run.

It's been a long time since I looked at this code. These days I use the python bindings to OpenCV for my computer vision needs.