A Hyperspectral Projector for 3D Imaging

A 3D point cloud of objects reconstructed by a hyperspectral stripe projector-based imaging system. The monochrome camera also captures spectral data for each point to provide not only the target’s form but also its material composition. (Source: Kelly Lab, Rice U.)

Stripes are in fashion this season at a Rice Univer­sity lab, where researchers use them to make images that plain cameras could never capture. Their compact Hyper­spectral Stripe Projector (HSP) is a step toward a new method to collect the spatial and spectral infor­mation required for self-driving cars, machine vision, crop monitoring, surface wear and corrosion detection and other appli­cations.

“I can envision this tech­nology in the hands of a farmer, or on a drone, to look at a field and see not only the nutrients and water content of plants but also, because of the 3D aspect, the height of the crops,” said Kevin Kelly, an associate professor of electrical and computer engineering at Rice’s Brown School of Engi­neering. “Or perhaps it can look at a painting and see the surface colors and texture in detail, but with near-infrared also see underneath to the canvas.”

Kelly’s lab could enable 3D spectroscopy on the fly with a system that combines the HSP, a monochrome sensor array and sophis­ticated programming to give users a more complete picture of an object’s shape and composition. “We’re getting four-dimen­sional information from an image, three spatial and one spectral, in real time,” Kelly said. “Other people use multiple modu­lators and thus require bright light sources to accomplish this, but we found we could do it with a light source of normal brightness and some clever optics.”

HSP takes a cue from portable 3D imaging techniques that are already in consumers’ hands and adds a way to pull broad spectral data from every pixel captured. This compressed data is reconstructed into a 3D map with spectral infor­mation that can incor­porate hundreds of colors and be used to reveal not only the shape of an object but also its material compo­sition. “Regular RGB (red, green, blue) cameras basically give you only three spectral channels,” Rice alumna Yibo Xu said. “But a hyper­spectral camera gives us spectra in many, many channels. We can capture red at around 700 nanometers and blue at around 400 nanometers, but we can also have band­widths at every few nanometers or less between. That gives us fine spectral resolution and a fuller understanding of the scene.”

“HSP simul­taneously encodes the depth and hyper­spectral measure­ments in a very simple and efficient way, allowing the use of a monochrome camera instead of an expensive hyper­spectral camera as typically used in similar systems,” said Xu. She developed both the hardware and reconstruc­tion software as part of her thesis in Kelly’s lab. HSP uses an off-the-shelf digital micromirror device (DMD) to project patterned stripes that look something like colorful bar codes onto a surface. Sending the white-light projection through a dif­fraction grating separates the over­lapping patterns into colors.

Each color is reflected back to the mono­chrome camera, which assigns a numerical grey level to that pixel. Each pixel can have multiple levels, one for every color stripe it reflects. These are recombined into an overall spectral value for that part of the object. “We use a single DMD and a single grating in HSP,” Xu said. “The novel optical design of folding the light path back to the same diffraction grating and lens is what makes it really compact. The single DMD allows us to keep the light we want and throw away the rest.”

These finely tuned spectra can reach beyond visible light. What they reflect back to the sensor as multi­plexed fine-band spectra can be used to iden­tify the material’s chemical composition. At the same time, distortions in the pattern are recon­structed into 3D point clouds, essentially a picture of the target, but with a lot more data than a plain snapshot could provide.

Kelly envisions HSP built into car headlights that can see the difference between an object and a person. “It could never get confused between a green dress and a green plant, because everything has its own spectral signature,” he said. Kelly believes the lab will eventually incor­porate ideas from Rice’s ground­breaking single-pixel camera to further reduce the size of the device and adapt it for compressive video capture as well. (Source: Rice U.)

Reference: Y. Xu et al.: A hyperspectral projector for simultaneous 3D spatial and hyperspectral imaging via structured illumination, Opt. Exp. 28, 29740 (2020); DOI: 10.1364/OE.402812

Link: Applied Physics, Rice University, Houston, USA

Speak Your Mind