Researchers perform non-line-of-sight ghost imaging with human vision

Researchers have shown that the computational imaging technique known as ghost imaging can be combined with human vision to image an object that can’t directly be seen by the person. The new work represents a step toward combining human intelligence with artificial intelligence.

“We believe that this work provides ideas that one day might be used to bring together human and artificial intelligence,” explained Daniele Faccio, Professor of Quantum Technologies, School of Physics and Astronomy, University of Glasgow. “The next steps in this work range from extending the capability to provide 3D depth information to looking for ways to combine multiple information from multiple viewers at the same time.”

Daniele Faccio from the University of Glasgow in the U.K. will present the new findings, titled “Non-Line-of-Sight (NLoS) Imaging and Imaging through Scattering Media,” at the Optica (formerly OSA) Imaging and Applied Optics Congress 11–15 July 2022. Faccio will present on Monday, 11 July, at 15:00 PDT.

Ghost imaging forms an image of an object by correlating a projected light pattern that interacts with the object and a reference pattern that does not. Although the researchers previously used human vision in a passive manner to perform ghost imaging, the new work uses the human visual system in an active role by having a person view the light patterns instead of a camera. The brain’s visual response is recorded and used as feedback for an algorithm that determines how to reshape the projected light patterns and reconstructs the final image.

“This is one of the first times that computational imaging has been performed by using the human visual system in a neurofeedback loop that adjusts the imaging process in real time,” said Faccio. “Although we could have used a standard detector in place of the human brain to detect the diffuse signals from the wall, we wanted to explore methods that might one day be used to augment human capabilities.”

The experimental setup used a projector to create light patterns on a cardboard cut-out that acted as the object being imaged. The transmitted light diffused onto a secondary white wall hidden from view by a dark wall. Thus, the observer could only see diffuse light reflected from the secondary white wall. Each light pattern was flickered at 6 Hz for two seconds, creating a signal in the observer’s visual cortex that was detected with a single-electrode electroencephalography (EEG) headset.

The researchers used the EEG signal to estimate the intensity of light transmitted by the object and diffused from the white wall, information that was fed into the neurofeedback loop used to reconstruct the image. If the EEG signal fell below a certain threshold, the researchers concluded that the light pattern didn’t overlap with the object and could thus be automatically removed, or carved out, in real time by the system.

The researchers showed that their technique could successfully reconstruct 16 x 16-pixel images of simple objects that could not been seen by the observer. They also demonstrated that the carving out process helped reduce the observation time needed for image reconstruction to about one minute.