Technology that can see what’s literally out of sight

The U.S. Defense Department’s Advanced Research Projects Agency (DARPA) has awarded a $4.4 million grant to researchers at the Morgridge Institute for Research and University of Wisconsin-Madison to design a “camera” that can see what’s literally out of sight.

Similar grants to explore the limitations and potential applications of scattered-light technology that can recreate scenes outside the human line of sight have been given to seven other universities.

The technology, pioneered by Morgridge imaging specialist Andreas Velten, uses pulses of scattered light photons that bounce through a scene and are recaptured by finely tuned sensors connected to the camera. Information from this scattered light helps the researchers digitally rebuild a 3D environment that is either hidden or obstructed from view.

Though in its infancy, the technology has generated excitement about potential applications in medical imaging, disaster relief, navigation, robotic surgery and even space exploration.

Velten is now collaborating with Mohit Gupta, assistant professor of computer sciences at UW-Madison (one of eight university teams receiving 2016 DARPA grants), to see how far they can take the technology. They are creating models where the light bounces six or more times to spot things that are outside the field of view.

“The more times you can bounce this light within a scene, a possible data you can collect,” Velten says. “Since the first light is the strongest, and each proceeding bounce gets weaker and weaker, the sensor has to be sensitive enough to capture even a few photons of light.”

Gupta, meanwhile, is developing algorithms that can better read the data and help recreate what the photons have hit.“The information we will get is going to be noisy and the shapes will be blob-like, not much to the naked eye, so the visualization part of this will be huge,” Gupta says. “Because this problem is so new, we don’t even know what’s possible.”

The first two years of the grant will be devoted to the pushing the limitations of this technique. The second two years will focus on hardware development to make field applications possible.

Some of the possibilities are enticing. For example, Gupta says it could be used for safety tests of jet engines, examining performance while the engines are running. It could also be used to probe impossible-to-see spaces in shipwrecks such as the Titanic. Velten currently has a NASA project examining whether the technology can be used to probe the dimensions of moon caves.