Musicjelly

I built interactive holographic audiovisual installations to allow people to make their own music. Users were able to use iPad interfaces, we also developed pioneering gesture based control systems with Kinect. The main challenges involved loading and playing back multiple synchronised high quality sound and videos on demand with low latency. Mostly written in  C++ with some C# for integration with Microsoft Kinect.

I also developed a 3D video filming system where input from multiple cameras was stitched together to create a 3D model of a subject. This involved a great deal of mathematical and analytical work, especially as custom compression techniques had to be developed, which required much research and analysis of different approaches. Computer Vision techniques were required to help calibrate and align the input of the cameras to create a single unified coordinate system. The main challenges involved stitching input from multiple cameras together to create a 3D model of a subject, custom compression techniques to compress 1.2Gb of data generated per of second film to a tenth of original size and developing high speed rendering techniques using the GPU. Written in C++ and OpenCL