A new paper presented in the Displays journal looks to biology and nature for a potential solution to the demand for ever-more advanced image fusion techniques, focusing on the development of image fusion algorithms based on the vision of rattlesnakes.
Study: Image fusion based on the rattlesnake visual receptive field model. Image Credit: Alexander Wong/Shutterstock.com
The image fusion process is a means of image processing whereby the same scene is acquired by various devices and then integrated into high-quality images using advanced computer technology. This continuously evolving field is the focus of a great deal of research, with researchers leveraging numerous mathematical approaches to improve image fusion algorithms.
There has been a range of previous research which seeks to explore biological imaging systems in image fusion technology, with the paper’s authors citing research on retina nerve cells, and, most notably, a previous study into pythons’ infrared and visual spectrum perception.
The rattlesnake is a fascinating animal that possesses infrared sensors located between the nostrils and its eye for light information. It is able to naturally combine these two sets of information, performing image fusion – an ability that is key to its ability to assess its surroundings, hunt prey, and avoid predators, even in dark environments.
A study by Hartline et al. in 1978 is highlighted, as this verified the existence of six bimodal cells in rattlesnakes, with specific neurons responsible for perceiving visible light and infrared that are present in the animal’s optic tectum. These cells are able to respond to either visible or infrared information or to both simultaneously – a system which the paper’s authors noted could be simulated via a series of mathematical models.
Using this innovative and ground-breaking research as the foundation of their work, the paper’s authors were able to develop and propose an effect-based pseudo-color image fusion method able to combine the infrared and visible spectrums into a useful image. This approach set out to imitate the working principle of rattlesnake vision, known as the Waxman fusion structure and leverages a visual neural network to achieve promising results.
Comparative experiments to evaluate the resulting images from the use of this algorithm highlight that – on a subjective visual inspection – the overall color presentation was good, and details were clearly visible by integrating the infrared and visible light spectrums.
More from AZoM: Biomimicry Offers Flexible Energy Storage Solution for Wearable Electronics
A series of evaluation indexes were used to objectively quantify these findings. For example, entropy was evaluated to determine the amount of information contained in the fusion image, while the average gradient provided insight into image clarity and fusion texture characteristics. Spatial frequency offered useful information on the overall activity of the image, while standard deviation provided details on the distribution of information and contrast – two factors linked closely with image quality.
The comparative experiment evaluated four groups of images, highlighting excellent performance in terms of average gradient and spatial frequency but less than ideal performance in terms of entropy and standard deviation.
By leveraging the rattlesnake’s bimodal imaging capabilities and simulating these using an image processing algorithm, the paper’s authors were able to begin developing a useful new approach to image fusion.
This enabled a rattlesnake bimodal cell mathematical model to be introduced, facilitating the acquisition of high-quality fusion images with natural color and detailed environmental information.
The ability to take different yet complementary image data and combine this into a series of useful images has a number of practical applications, and as such, image fusion is a key technique in a vast array of fields, including safety, remote sensing, astronomy, medical and logistics applications.
By expanding beyond conventional mathematical approaches and looking at evolution and biology for inspiration, it will be possible to continue to expand and refine image fusion methods – like the one presented here – to expand their applicability, improve their reliability, and enhance their accuracy.
Reference
Y. Wang, H. Liu, W. Xie, S. Wang, Image fusion based on the rattlesnake visual receptive field model, Displays (2022), doi: https://www.sciencedirect.com/science/article/pii/S014193822200018X?via%3Dihub
Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.