Finally it's here: a Minority Report-like interface using the Kinect sensor. The MIT Computer Science and Artificial Intelligence Laboratory has crated an interface that looks very similar to the one used in the movie Minority Report. See the video below:

The developers write:

This is a graphical interface inspired by the movie "Minority Report". It uses the Kinect sensor from Microsoft, and the recently released libfreenect driver for interfacing with the Kinect in linux. The graphical interface and the hand detection software were written at MIT to interface with the open source robotics package 'ROS', developed by Willow Garage. The hand detection software showcases the abilities of the Point Cloud Library (PCL), a part of ROS that MIT has been helping to optimize. The hand detection software is able to distinguish hands and fingers in a cloud of more than 60,000 points at 30 frames per second, allowing natural, real time interaction.

The source code is available at: http://www.ros.org/wiki/kinect and http://www.ros.org/wiki/mit-ros-pkg

The software library MRPT now implements a common C++ library for the Kinect sensor. The library works both on top of the Linux/Mac OS X driver and the Windows driver. The advantage of this approach is that developers can now write software using one unified API that controls a Kinect sensor on all platforms.

Here is a video of a demo application can constructs a 3D environment with the Kinect output:

The developers promise 30 frame per second performance on both Linux and Windows. For more information and downloads, check http://www.mrpt.org/Kinect_and_MRPT

We wrote about hand and finger tracking previously, but it seems that the sofware has evolved quite a bit. See the demo below of a finger tracking demo:

As you can see, the method seems to be working quite flawless. Here's a video of a demo application built on top of this:

For more updates, follow the RobbeOfficial YouTube channel. We'll post an update if the source code is released.

The people of UC Berkerly EECS department have created an interesting piece of software that uses the Kinect sensort to fly a device and avoid objects. The device flies autonomically and pauses when an object is placed in front of it. See the video below:

From the video description:

This work is part of the STARMAC Project in the Hybrid Systems Lab at UC Berkeley (EECS department). http://hybrid.eecs.berkeley.edu/

Researcher: Patrick Bouffard
PI: Prof. Claire Tomlin

Our lab's Ascending Technologies [1] Pelican quadrotor, flying autonomously and avoiding obstacles.

The attached Microsoft Kinect [2] delivers a point cloud to the onboard computer via the ROS [3] kinect driver, which uses the OpenKinect/Freenect [4] project's driver for hardware access. A sample consensus algorithm [5] fits a planar model to the points on the floor, and this planar model is fed into the controller as the sensed altitude. All processing is done on the on-board 1.6 GHz Intel Atom based computer, running Linux (Ubuntu 10.04).

A VICON [6] motion capture system is used to provide the other necessary degrees of freedom (lateral and yaw) and acts as a safety backup to the Kinect altitude--in case of a dropout in the altitude reading from the Kinect data, the VICON based reading is used instead. In this video however, the safety backup was not needed.

[1] http://www.asctec.de
[2] http://www.microsoft.com
[3] http://www.ros.org/wiki/kinect
[4] http://openkinect.org
[5] http://www.ros.org/wiki/pcl
[6] http://www.vicon.com

Stephen Hobley wrote an excellent article on the Kinect sensor and the hacking efforts. He explains why it's not trivial to get body recognition working like the Xbox 360 does. Getting the RGB and depth video stream from the Kinect is one thing, but translating this to human detection using image processing software is a lot harder.

Check out the complete article at http://www.stephenhobley.com/blog/2010/12/04/what-the-kinect-sensor-actually-...

We haven't seem someone before trying this: rotating a 3D view of the Kinect sensor with hand gestures. Check the video below for a demo:

It's unknown if and when developer Jeppe Walther will release the source code of this project.

In a EU-funded project by EPFL, several students have created software that allows real-time people detection using multiple Kinects. The Kinect sensors are connected to a single laptop, and detection work both at daytime and nighttime. The source code is not available yet, but check the video below:

DaVinci is a Microsoft Surface application that "blurs the lines between the physical and virtual world by combining object recognition, real-world physics simulation and gestural interface design". The developers have added Kinect support lately, and the result is a stunning physics playground based on gestures. See the video below:

The project looks very mature, it's unsure when this will be released. More info at http://emergingexperiences.com/2010/12/davinci-goes-touchless-with-xbox-kinect/

This is a surprising hack using the Kinect: an invisibility cloack. The video is pretty long, the interesting part starts around 1:30 minutes. It's not really clear how this hack works (there is no source available), but the results are quite amazing:

The same Dustin O'Connor as in the previous post is working on software that allows full mouse control using the Kinect in Mac OS X. The source is not available yet, but the video below looks very promising:

As you can see, the mouse point doesn't jump around a lot and looks quite stable. We are very interesting in testing this on our own installation, we'll post an update if the source code becomes available.