Share HoloLens Experience with Others Using Spectator View Tool, New Cognitive Services APIs:Vision

Microsoft documentation helping build a spectator view camera for HoloLens. A way for people to see what someone sees when wearing HoloLens from a third-person perspective. Cognitive Services APIs: Vision.

Share online:

A new Microsoft documentation to help you build a spectator view camera for HoloLens, which allows an audience to view what a Microsoft HoloLens wearer is seeing and experiencing is available as on Monday Feb 13.

Spectator View Camera, is an exciting new tool that help others see what you see in HoloLens. "When we created HoloLens, we knew that we needed to create a way to connect the experience of someone wearing a HoloLens with that of an audience (whether that audience is 1 person or 100 people)," writes Brandon Bray.

Spectator View Tool for Microsoft HoloLens
Spectator View Tool for Microsoft HoloLens

"Since a HoloLens can only be worn by one person, how do you open the view so everyone can see what that person sees? To solve this problem, Mixed Reality Capture (MRC) came to aid in visualization of holograms that the wearer is seeing from a first-person perspective," Bray said.

"MRC is built right into the device so it's always there and ready to use. It's a very simple way to capture what is seen from the device so that your audience can follow along when you are giving a demo or enjoying a HoloLens app you want others to see," he said.

In essence, a spectator view camera can be used in three specific scenarios: "Capturing photos of a mixed reality scene, Shooting video of a holographic environment in action, and Live streaming holographic content to an audience."

In simpler terms, MRC offers a first-person perspective of the augmented reality experience, while spectator view uses a DSLR in conjunction with software to provide a third person view of the augmented construction, and at a higher quality. In essence, it allows audience to see what people wearing HoloLens at events, though at a fraction of the cost of the setup used.

Microsoft also shared a peek into how what the world will look like once your spectator view camera setup is complete:

With all of the setup, "the DSLR will basically record its surroundings with inclusion of any holograms you have set up in the space, presenting the ability to both stream live and record video, alongside taking still pictures," added Bray.

To create your own spectator view camera, head over to GitHub to see the step-by-step guidance documentation for getting started. In the video below, Microsoft walk through the steps to bring a spectator view camera to life—see it below.

Here's what you'll have to do:

  • Acquire the proper hardware and a DSLR camera with HDMI output.
  • Mount one HoloLens to the camera with the mounting bracket.
  • Once mounted, you will be able to connect your spectator view camera to your PC. The HoloLens will communicate to your PC via Wi-Fi – most gaming and workstation PCs will be powerful enough to handle the job.
  • Your app will need to be a shared experience, and can run on both a HoloLens and a desktop – leveraging the Universal Windows Platform. The online Holograms 240 class will help you learn how to create a shared experience.
  • Complete some calibrations to ensure your HoloLens and DSLR camera are aligned.
  • You are now ready to share your holographic creations, he said.

Today, Microsoft also shared new "Cognitive Services APIs," are a bridge allowing web and UWP developers to use the resources of major AI research to solve developer problems. It is grouped into following five categories:

  • Vision—analyze images and videos for content and other useful information.
  • Speech—tools to improve speech recognition and identify the speaker.
  • Language—understanding sentences and intent rather than just words.
  • Knowledge—tracks down research from scientific journals for you.
  • Search—applies machine learning to web searches.
Cognitive Services APIs: Vision
Cognitive Services APIs: Vision

For those who are not aware, "Cognitive Services are a set of machine learning algorithms that Microsoft has developed to solve problems in the field of Artificial Intelligence (AI)." "The goal of Cognitive Services is to democratize AI by packaging it into discrete components that are easy for developers to use in their own apps." "Web and Universal Windows Platform developers can consume these algorithms through standard REST calls over the Internet to the Cognitive Services APIs," explained Microsoft.

To get started, the first thing you need to do is register at the Cognitive Services site and request a key for Computer Vision Preview (by clicking on one of the "Get Started for Free" buttons.

Also, check out these resource to learn AI / Machine Learning, and see the Code samples.

Additionally, watch this video below demonstrating Congnitive Services: