Mantis Shrimp-Camera for Self-Driving Cars

A new bioinspired camera could improve the ability of cars to spot hazards in challenging imaging conditions such as the foggy conditions shown in this video. The first image panel was acquired with a regular camera, and the other shows the degree of polarization and angle of polarization acquired with the new camera. (Source: V. Gruev, UIUC)

Inspired by the visual system of the mantis shrimp researchers have created a new type of camera that could greatly improve the ability of cars to spot hazards in chal­lenging imaging conditions. The new camera accom­plishes this feat by detecting a property of light known as polari­zation and featuring a dynamic range about 10,000 times higher than today’s commercial cameras. Dynamic range is a measure of the brightest and darkest areas a camera can capture simul­taneously. With these, the camera can see better in driving condi­tions such as the transition from a dark tunnel into bright sunlight or during hazy or foggy condi­tions.

The researchers say the new camera would enable cars to detect hazards, other cars and people three times farther away than color cameras used on cars today. “In a recent crash involving a self-driving car, the car failed to detect a semi-truck because its color and light intensity blended with that of the sky in the background,” said research team leader Viktor Gruev of the Univer­sity of Illinois at Urbana-Champaign. “Our camera can solve this problem because its high dynamic range makes it easier to detect objects that are similar to the background and the polari­zation of a truck is different than that of the sky.”

In addition to auto­motive appli­cations, the researchers are exploring using the cameras to detect cancerous cells, which exhibit a different light polari­zation than normal tissue, and to improve ocean explo­ration. “We are beginning to reach the limit of what tradi­tional imaging sensors can accomplish,” said Missael Garcia. “Our new bio­inspired camera shows that nature has a lot of interesting solu­tions that we can take advantage of for designing next-gene­ration sensors.”

Mantis shrimp, a grouping that includes hundreds of species worldwide, have a loga­rithmic response to light intensity. This makes the shrimp sensitive to a high range of light inten­sities, allowing them to perceive very dark and very bright elements within a single scene. To achieve a similarly high dynamic range for their new camera, the researchers tweaked the way the camera’s photo­diodes convert light into an electrical current. Instead of operating the photo­diodes in reverse bias mode the researchers used forward bias mode. This changed the elec­trical current output from being linearly propor­tional to the light input to having a loga­rithmic response like the shrimp.

For the polari­zation sensi­tivity, the researchers mimicked the way that the mantis shrimp integrates polarized light detection into its photo­receptors by depositing nano­materials directly onto the surface of the imaging chip that contained the forward biased photo­diodes. “These nano­materials essentially act as polari­zation filters at the pixel level to detect polari­zation in the same way that the mantis shrimp sees polari­zation,” said Gruev.

Although tradi­tional imaging sensor fabri­cation processes can be used to make the sensors, they are not optimized for making photo­diodes that operate in a forward bias. To compensate, the researchers developed addi­tional processing steps to clean up the images and to improve the signal to noise ratio. After testing the camera under different light inten­sities, colors and polarization conditions in the lab, the researchers took the camera into the field to see how well it operated in shadows as well as in bright conditions. “We used the camera under different driving lighting conditions such as tunnels or foggy condi­tions,” said Tyler Davis, a member of the research team. “The camera handled these chal­lenging imaging condi­tions without any problems.”

The researchers are now working with a company that manu­factures air bags to see if the new camera’s high dynamic range and polari­zation imaging capa­bility can be used to better detect objects to either avert a col­lision or deploy the air bag a few milli­seconds earlier than is currently possible. The researchers also received funding to use the new imaging system to make small GoPro-like cameras that could be used to explore the ocean. While GPS systems such as those in cell phones do not work under water, the new camera’s polari­zation detection capa­bility allows it to use the polari­zation of sunlight in water to calcu­late location coor­dinates. In addition, the camera’s high dynamic range could be used to record high quality images under water.

“We are coming full circle by taking the camera, which was inspired by mantis shrimp, to different tropical oceans to learn more about how this shrimp behaves in its natural habitat,” said Gruev. “They live in shallow waters and bury them­selves under corals or in little burrow. This creates a chal­lenging high dynamic range imaging situa­tion because there’s a lot of light in the water but dim condi­tions inside the holes.” (Source: OSA)

Reference: M. Garcia et al.: Bioinspired polarization imager with high dynamic range, Optica 5, 1240 (2018); DOI: 10.1364/OPTICA.5.001240

Link: Biosensors Lab, Dept. of Electrical and Computer Engineering, University of Illinois, Urbana-Champaign, USA

Speak Your Mind