NASA Scientists Control Robotic Arm With Kinect And Oculus Rift

By Joelle Renstrom | Updated

This article is more than 2 years old

NASA JACORobots in space — two great tastes that taste great together. They might go to Mars ahead of the humans to set up facilities, 3-D printed robotic spiders might build spacecraft, and robots might make isolated astronauts less lonely. Yet another robotics advancement at NASA pairs humans and robots, allowing humans to control the actions of robotic counterparts using a good ol’ Microsoft Kinect and something called the Oculus Rift.

The Kinect sensor provides the position tracking, while the Oculus Rift virtual reality headset allows for rotational tracking and the first-person experience one gets when playing a game in virtual reality. When the user has the view he needs, he can perform tasks which control the real-time movements of robotic arm. NASA has been using a JACO robotic arm developed by Kinova, a Canadian company that specializes in rehabilitation and research. The arm has three fingers, six degrees of freedom, and is designed to represent a “new generation of lightweight portable robotic manipulators.”

NASA arm with kinect

NASA’s Jet Propulsion Laboratory will soon move from controlling a robotic arm to controlling an entire robotic system, namely the Robonaut 2 currently aboard the ISS. While the ISS may not be a home base for as many humans as we may have once thought it would, it is a base for robots and for piloting technologies that could facilitate space exploration. A human-controlled robot would be able to perform difficult tasks (although Gravity wouldn’t have been nearly as compelling had there been a robot or two out there fixing the Hubble).

Oculus Rift

Gaming technology has proven particularly useful to NASA. Last year (and by that I mean 2013 — it still seems a bit strange), NASA used the Oculus Rift headset after playing with one at a Penny Arcade eXpo (PAX) gaming festival. They linked the Rift with a treadmill and used images sent back from Mars by the Curiosity Rover to see what it might be like to walk around the surface of Mars. This is a brilliant use of virtual reality gaming systems — who ever said the virtual environment had to be entirely fabricated? I could see such a system in science museums as well, giving visitors the opportunity to take a virtual walk on another planet, much as a visit to Epcot Center can feel like a virtual visit to another part of the world. It also provides a possible way to virtually accompany astronauts or Mars colonists as they explore parts unknown. Video game naysayers clearly haven’t been to space, but could one day get a taste of the cosmos by embracing one of the systems. How I love irony!