Rock & Rails: Microsoft Research Extending Multi-touch Interactions with Shape Gestures

A Microsoft researcher, Hrvoje Benko, is working on methods of recognizing shapes formed by hands and equating those with spatially-consistent gestures. Putting the side of your hand down like a wall forms a straight line that could be used for a boundary, cropping, or "pushing" objects. Forming an O with your hand could automatically call […]

A Microsoft researcher, Hrvoje Benko, is working on methods of recognizing shapes formed by hands and equating those with spatially-consistent gestures. Putting the side of your hand down like a wall forms a straight line that could be used for a boundary, cropping, or "pushing" objects. Forming an O with your hand could automatically call up the magnification loupe, and so on.

"Direct touch manipulations enable the user to interact with the on-screen content in a direct and easy manner closely mimicking the spatial manipulations in the physical world. However, they also suffer from well-known issues of preci-sion, occlusion and an inability to isolate different degrees of freedom in spatial manipulations.

We present a set of interactions, called "Rock & Rails," that augment existing direct touch manipulations with shape-based gestures, thus providing on-demand gain control, occlusion avoidance, and separation of constraints in 2D manipulation tasks.

Using shape gestures in combination with direct-manipulations allows us to do this without ambiguity in detection and without resorting to manipulation handles, which break the direct manipulation paradigm. Our set of interactions were evaluated by 8 expert graphic designers and were found to be easy to learn and master, as well as effective in accomplishing a precise graphical layout task," reads the description.

The video shows the tech being demonstrated on a Surface, which uses a different detection method than your average smartphone or tablet -- it can detect shapes far more easily.

[Source: Microsoft Research]

You can keep track of this research paper and other projects on to Hrvoje’s page.