Today I came across a great post by NASA, talking about their collaboration with Microsoft Hololens to develop Sidekick, a new project using commercial technology to empower astronauts aboard the International Space Station (ISS).
Project Sidekick will use Microsoft HoloLens to provide virtual aid to astronauts working off the Earth, for the Earth. NASA mentioned on their website that a pair of the devices is basically scheduled to launch on SpaceX’s seventh commercial resupply mission to the IS station on June 28. To test Sidekick, NASA and Microsoft engineers went on board NASA’s Weightless Wonder C9 jet.
In summary, Sidekick with its holographic computing aims at enabling station crews with support when and where needed. This new capability could reduce crew training requirements and increase the efficiency at which astronauts can work in space.
Aboard NASA’s Weightless Wonder C9 jet, the teams from both sides went on testing Project Sidekick and the Microsoft HoloLens to ensure they function as expected in free-fall, in advance of their delivery to the microgravity environment of the space station.
Reading the post by NASA, I learned that Sidekick has two modes of operation. The first is “Remote Expert Mode,” which uses Skype, product of Microsoft, to allow a ground operator to see what a crew member in space are seeing, provide real-time guidance, and draw annotations into the crew member’s environment to coach him or her through a task. Until now, crew members have relied on written and voice instructions when performing complex repair tasks or experiments. But Microsoft Hololens, can change of all of that and revolutionize it!
The second mode is “Procedure Mode,” where by it augments standalone procedures with animated holographic illustrations displayed on top of the objects with which the crew is interacting. This type of capability could actually reduce the amount of training that future crews will need and could be an invaluable resource for missions deep into our solar system, where communication delays complicate difficult operations.
It is an exciting journey and NASA expects that astronauts on the station will first use Sidekick by the end of the year.
Sidekick also will first be used very soon where it will be evaluated during the NASA Extreme Environment Mission Operations (NEEMO) 20 expedition set to begin July 21 when a group of astronauts and engineers live in the world’s only undersea research station, Aquarius, for two weeks. The Aquarius habitat and its surroundings offer a convincing analog for space exploration.
The Sidekick project is not the only collaboration between Microsoft and NASA, it is a part of a larger partnership between the two to explore applications of holographic computing in space exploration. Earlier this year, they announced a collaboration to develop software called OnSight, a new technology that will enable scientists to work virtually on Mars using the same HoloLens technology. NASA’s Jet Propulsion Laboratory (JPL) in Pasadena, California, led the development of Sidekick and OnSight. You can read more on the official post by NASA, here is the link to that post.
I will leave you with this cool video which shows members from Microsoft Hololnes team testing project Sidekick on board the Weightless Wonder C9 jet.