Video: Peeking Behind the 'HoloDesk' Microsoft Research Project

HoloDesk is an innovative project from the Sensors and Devices Group, part of the Computer-Mediated Living research area at Microsoft Research. "HoloDesk uses a Kinect depth camera to enable the manipulation of virtual, 3-D objects with your hands. It remains a research project at this stage, but the potential to make our interactions with computers […]

HoloDeskHoloDesk is an innovative project from the Sensors and Devices Group, part of the Computer-Mediated Living research area at Microsoft Research.

"HoloDesk uses a Kinect depth camera to enable the manipulation of virtual, 3-D objects with your hands. It remains a research project at this stage, but the potential to make our interactions with computers more natural is staggering, which is what has people all abuzz--even in the absence of many details." Microsoft's Rob Knies got in touch with Otmar Hilliges, a researcher in the Sensors and Devices Group, to gain some insight into just how the prototype works.

"We're using a regular LCD screen to render perspective graphics that are reflected off a semi-silvered mirror," he responded, referring to a semi-opaque mirror that partially reflects an image from the front and allows an image to shine through from behind," explains Hilliges.

"The user looks through the semi-silvered mirror so that the scene behind the mirror and the reflected graphics appear combined."

The use of a Kinect depth camera accelerated the pace of the research considerably.

"We've been trying to simulate human grasping within physics simulations for quite a while," Hilliges said. "Previously, the input data weren't rich enough to allow for the 3-D interaction fidelity that HoloDesk provides. The Kinect is almost perfectly suited for this sort of task, so we started working on this project as soon as it became available to us--a couple of months before public availability."

The researchers involved are writing a paper to document their approach, but while that awaits publication, they already are well aware of what they have wrought. I asked Hilliges what makes HoloDesk unique.

"The possibility to directly reach into a 3-D scene," he responded, "and to directly touch 3-D objects that respond to this natural user input in a physically plausible manner. Modeling this sort of input was the main contribution of the research around HoloDesk."