So we faced an interesting problem coming out of our alpha, where we realised that we have to decide on how we are going to represent users interaction. There were compiling errors trying to integrate OpenCV into our build. I spent this week looking into Kinect body puppet. Found a really good example, shown below.
There are infinitely many directions to take, from morphing shapes to objects combining to create shapes, etc. I explored the puppet idea because it easy makes fit into our design style and has seamless experience for the users. I couldn’t get bone/joint rotations information, but I found a workaround that should work.