NASA to empower astronauts with Microsoft HoloLens

WASHINGTON: NASA and Microsoft are teaming up to develop Sidekick, a new project using commercial technology to empower astronauts aboard the International Space Station (ISS).

Sidekick uses Microsoft HoloLens to provide virtual aid to astronauts working off the Earth, for the Earth, NASA said in a prepared statement.

The goal of Sidekick is to enable station crews with assistance when and where they need it. This new capability could reduce crew training requirements and increase the efficiency at which astronauts can work in space.

Sidekick has two modes of operation. The first is Remote Expert Mode, which uses Skype to allow a ground operator to see what a crew member sees, provide real-time guidance, and draw annotations into the crew member's environment to coach him or her through a task. Until now, crew members have relied on written and voice instructions when performing complex repair tasks or experiments.

The second mode is Procedure Mode, which augments standalone procedures with animated holographic illustrations displayed on top of the objects with which the crew is interacting. This capability could lessen the amount of training that future crews will require and could be an invaluable resource for missions deep into our solar system, where communication delays complicate difficult operations.

After the devices arrive at the space station, crew members will test and verify the software and hardware functionality in a standalone mode. A second set of devices to be delivered on a future mission will test and verify Sidekick functionality with network connectivity to test the Remote Expert Mode. NASA expects astronauts on the station will first use Sidekick by the end of the year.

Sidekick also will be used and evaluated during the NASA Extreme Environment Mission Operations (NEEMO) 20 expedition set to begin July 21 when a group of astronauts and engineers live in the world's only undersea research station, Aquarius, for two weeks.

"HoloLens and other virtual and mixed reality devices are cutting edge technologies that could help drive future exploration and provide new capabilities to the men and women conducting critical science on ISS," said Sam Scimemi, Director, ISS program, Nasa Headquarters. "This new technology could also empower future explorers requiring greater autonomy on the journey to Mars."

Alex Kipman, Technical Fellow, Windows and Devices Group, Microsoft, said, "Sidekick is a prime example of an application for which we envisioned HoloLens being used-unlocking new potential for astronauts and giving us all a new perspective on what is possible with holographic computing."

It must be noted that NASA and Microsoft engineers tested Project Sidekick and the Microsoft HoloLens aboard NASA's Weightless Wonder C9 jet to ensure they function in free-fall in advance of their delivery to the microgravity environment of the space station.