Menu
Since the release of the Microsoft Kinect for Xbox 360, the success of the Nintendo Wii, and its many applications in VR, motion tracking through the use of RGB-D cameras, able to capture both colors and depth information, gained a lot of interest outside of the games industry. Even more so: these days you may find more non-gaming applications of motion tracking hardware than motion tracking-based games. As rapid prototyping experts, we’ve had the opportunity of working with this technology and seeing it evolve over the past decade.
Our expertise goes back to 2010, when the first Kinect controller was released for Xbox 360. Due to lack of an official SDK, we used OpenNI to work with the innovative technology for clients including Walt Disney Imagineering R&D. Later, after Microsoft released the Kinect for Windows, this became a lot more accessible. Over the past ten years, we’ve had the pleasure of working with many more motion tracking cameras, ranging from the Kinect V2 to the Leap Motion, ZED Stereo Camera and Azure Kinect. Since the proof is in the pudding, we believe it’s best to illustrate the versatility of this technology with some concrete examples.
1. Transferring you to a Different Place in the Immersion Cube
The following project is one we developed for 9yards, a London based creative agency that pioneers tailor made interactive, virtual, and experiential engagements for their clients. And pioneering they sure do – this is the same company for which we did our first VR project.
When it comes to motion tracking cameras, the project we wanted to focus on for this very article was called Immersion Cube. In this installation, users stand in a big cube that offers interactive visualizations based on the location of the person in the cube. We used the Microsoft Kinect to determine the position of the subject anywhere within the installation.
In the Immersion Cube project, a physical prototype in our client's workshop near London was combined with the software prototype we developed in Wetteren.
For this project, we worked side by side with Voodoo Animation Studios who worked on the concept, early visual mockups, the hardware setup, and final graphics. Make sure to check out the video below for an overview of the project, as we were able to make this footage available for the first time.
2. Studying Behavioral Avoidance with YIKES
If you’ve visited our blog before, chances are great you’ve read about YIKES before. In this project for the Yale Child Study Center, we use Microsoft’s Kinect technology to detect and measure behavioural avoidance. For this, we initially used the original Kinect controller, but upgraded to the Kinect 2 because of its updated hardware and expanded features. Find out more about YIKES in our press release on the project.
YIKES, a prototype used to measure behavioral anxiety in a playful way, was our first project using a motion tracking camera.
3. Getting a Move On in Stealth Game
For the wE-MOVE project, with a consortium which included Ghent University’s Department of Rehabilitation Sciences, we built a prototype that utilizes the Microsoft Kinect as a controller. Stealth Game is an endless runner, not unlike mobile games such as Temple Run and Subway Surfers in which you swipe left, right, down, or up in order to avoid obstacles. Stealth Game applies the same type of gameplay, but instead of swiping, you have to physically move a certain way to avoid objects on your path. You can find more information on the project in this blog post about 7 different projects in health research.
In this prototype, we used the Kinect in combination with a heart rate sensor to create an exercise game that implements interval training.
Other Notable Projects Using Motion Tracking Cameras
- Testing the Waters for Walt Disney Imagineering: For Walt Disney Imagineering R&D, we developed a series of prototypes using the Microsoft Kinect, helping them test various ideas for the Disney amusement parks.
- The Future is Now with Holographic Conferencing: In this prototype for an undisclosed client, we brought to life the ultimate science fiction dream of putting people in one room without physically bringing them together. We worked with the Stereolabs ZED camera in order to capture the participants of the call in a point cloud and used the HoloLens to show the person who isn’t physically there.
- Prototyping for Google: As part of our work on an undisclosed prototype for Google Research, we used their own technology BodyPix, and integrated it into Unity. BodyPix is a really interesting technology as it can use a regular RGB camera and recognize and track the skeleton of several users, similar to what the Kinect is able to achieve with more expensive hardware. They do this using the power of artificial intelligence through TensorFlow.
- Low Budget Mocap Recording and Processing: There are many pictures floating around showing off the expensive motion capturing (mocap) sessions done in high tech studios for game developers and Hollywood productions. Thanks to iPi Soft there’s also a cheaper, less complex alternative. For one of our clients, we leveraged this technology to prototype a motion capturing and recording solution utilizing the Microsoft Kinect as the dedicated motion capturing camera.
Experience the Possibilities for Yourself
The possibilities with the ever-improving motion cameras are endless. Inspired by these examples and looking to test some of your own ideas? Feel free to get in touch, we gladly help you with our experience and expertise.