At the Microsoft’s annual TechFest this week, Microsoft Researchers are showing off their Kinect-centric projects along with other projects. Microsoft Research showing off 154 demos and displays of the latest work from MSR labs around the world including:
New Experiences in Search explores ways for people to experience search that are complementary to fast, relevant search in response to queries. This project component includes an organic kind of search that presents results that grow over time, drawing attention to the things about which you are most passionate
Wearable Multitouch Projector allows a user to use their hands, arms or the wall as graphical interactive surfaces. As you can see in the video, this is still very much at the prototype stage but the potential is clear – with anything you can do on today’s mobile devices, potentially being in the palm of a hand.
Video: Wearable Multitouch Projector
FetchClimate! Building a Geographical Web Service
FetchClimate!, an intelligent, scalable, Windows Azure-based climate-data-retrieval service, can be used through a Silverlight Web interface or from inside any .NET program, and can provide climate information, data set location, negotiate permissions, download huge files, make sense of file formats, filter, interpolate and regrid.
Automatic Text Pop-Up for Web Images
Bing home page provides teaser captions for an interesting image in the form of Bing “cherries.” This project for Web Images application automatically generates similar text descriptions for a large fraction of the most popular images on the Web.
Natural User Interface for Polling Students in a Classroom project delivers a new, low-cost technique for instantly polling students in the classroom. Initial trials in schools in Bangalore, India, show the system is as accurate as a written test, as fast as a show of hands, and at least 10 times cheaper than alternative electronic solutions.
Bing-Enabled Azure Data Services for Enterprises showcases two data services: An entity-synonym service that uses search-query logs to discover different ways people refer to any given entity, and an entity-augmentation service that uses search-crawled data to discover important attributes of a given set of entities and to auto-fill their values on those attributes.
Kinect-based projects — ranging from new webcam prototypes to projectors enabling new kinds of virtual/augmented reality showed off during the opening days.
Gesture Recognition with Next-Generation Webcam
This project presents next-generation webcam hardware and software prototypes. The new prototype webcam has an extremely wider view angle than traditional webcams and can capture stereo movie and high-accuracy depth images simultaneously.
“Users can chat with stereoscopic video. Accurate depth-image processing can support not only all Kinect scenarios on a PC, but also a gesture-control user interface without a touch screen,” according to Microsoft’s write-up. “Besides computer vision, the webcam includes a hardware accelerator and a new image-sensor design. The cost of the design is similar to that of current webcams, and the webcam potentially could be miniaturized as a mobile camera,” Microsoft added.
Beamatron augmented-reality concept that combines a projector and a Kinect camera on a pan-tilt moving head. The moving head can place the projected image almost anywhere in a room, while the depth camera enables the correct warping of the displayed image for the shape of the projection surface. How could this be used in the real world? “A projected virtual car can be driven on the floor of the room but will bump into obstacles or run over ramps,” the Redmondians wrote.
SpatialEase is an Xbox 360 Kinect game for learning the language of space using ’embodied’ learning that connects language with thought and action. From the description: “The learner must quickly interpret second-language commands, such as the translation of ‘move your left hand right,’ and move his or her body accordingly.
Kinect in the dark: “Kinect technology can open up new interactions in the dark, for example helping us to ‘feel’ an invisible shape through sound feedback,” Microsoft researchers explained. There’s an accompanying dimly-lit video clip that is meant to highlight what’s going on with this research.
Shake n’ Sense: A Microsoft Cambridge project that looks to mitigate interference when two or more Kinect cameras point at the same scene. It makes use of mechanical augmentation of the Kinect and doesn’t require modification of the Kinect firmware, host software or inner guts.
Also, at the TechFest, Robin Angotti, an associate professor of math education at the University of Washington-Bothell, along with two computer science students — Jebediah Pavleas and Jack Chang, have created a custom Kinect app to help teach students various functions of mathematics such as distance, acceleration, and velocity by letting them plot these equations on a graph in real time using Kinect and their bodies rather than just computing an equation with a pencil on graph paper.
In Loudon County Public Schools, teachers are using “Kinect Sports” to help teach math concepts such as patterns versus randomness, probability, and angles through bowling. Third-graders use Kinect Fun Labs Kinect Googley Eyes gadget to give a presentation on lessons they have learned, and fourth-graders use Avatar Kinect to produce late-night shows and commercials, strengthening their language arts skills.
A fifth grade class conducted “Reading Idol” shows using Avatar Kinect to strengthen their reading skills; and a fourth-grade class used Kinect to conduct a videoconference simulation with Challenger Learning Center, a NASA-sponsored organization, and plans to hold additional conferences with students in Romania to broaden their cultural awareness.
Avatar Kinect uses motion tracking in Kinect plus facial recognition to give each student his or her own avatar, a real-time on-screen representation of the student, which mimics the way that the participant smiles, nods, speaks and gestures.
In the Los Angeles Unified School District, fourth-grade teachers at Middleton Elementary are using NAMCO BANDAI Games America Inc.’s “Body and Brain Connection” for Kinect for Xbox 360 to teach math lessons on angles, probability and arithmetic.
In this demo, you’ll see what the future of 3D facial animation holds and how we can get from our current state to a near perfect simulation all created digitally.