Introduction of hyperspectral imaging technology

1. Basic definition of spectroscopy

Spectroscopy is a method to study matter or energy by using the phenomena of light, sound or particles emitted, absorbed or reflected by matter. It is generally defined as the study of the interaction between electromagnetic waves of different wavelengths and matter. Spectroscopy is frequently used in physics and analytical chemistry to identify substances by emitting or absorbing electromagnetic waves. The graph of the interaction is called the spectrum, or sometimes the optical spectrum.

Spectroscopy is a university discipline. It has been more than 300 years since its birth. In the past 100 years, there have been more than 34 Nobel prizes related to spectroscopy. Spectroscopy related instruments, equipment and even intelligent hardware have become indispensable in our daily life.

There are many spectroscopy classifications and different technical paths, including Fourier transform spectroscopy, Raman spectroscopy, fluorescence spectroscopy, terahertz, spectrometer and so on.

2. Spectral imaging technology

The light source emits light of various frequencies (different wavelengths), which irradiates the object. Due to the physical properties of the material on the surface of the object, some of the light is absorbed by the surface of the object, and the other part is reflected. The mechanism is that different molecules, atoms and ions in the substance correspond to different energy levels with different characteristic distributions and produce transitions under the spectrum of a specific frequency, thereby causing spectral emission and absorption of different wavelengths, thus producing different spectral characteristics

Fig. 1 simple spectral curve of vegetable leaves under sunlight

Under the same conditions, the spectral characteristics of different substances (corresponding to different atoms, molecules or molecular groups) have uniqueness and some characteristics. According to these characteristics, the measured object can be analyzed.

The traditional spectral technology analyzes the object by self luminescence of the object to be measured or interaction with the light source. From the perspective of space, most of the traditional spectral technology is aimed at a single point position, that is, a single point spectrometer. Spectral imaging is a combination of spectral technology and imaging technology, combining spectral resolution and graphic resolution, resulting in surface spectral analysis in spatial dimension, which is the current multispectral imaging and hyperspectral imaging technology.

1. Multispectral and hyperspectral

The essence of spectral imaging technology is to make full use of the absorption or radiation characteristics of matter to different electromagnetic spectra, and to add one-dimensional spectral information on the basis of ordinary two-dimensional spatial imaging. Imaging spectrum can obtain image information and spectral information of pixels at the same time. Multispectral imaging and hyperspectral imaging technologies are introduced according to different spectral resolutions;

Multispectral Technology: the number of target bands is between 3 and 30 (usually greater than or equal to 3);

Hyperspectral imaging: the number of target bands is between 100 and 300, and the spectral resolution is generally finer.

Fig. 2 Comparison of RGB imaging, multispectral imaging and hyperspectral imaging (2)

Hyperspectral imaging is a new technology based on spectral analysis. It collects hundreds of images with different wavelengths for the same spatial area. The collected data form a so-called hyperspectral cube. Usually, the abscissa and ordinate of the image represent the wavelength and intensity of the spectrum respectively. The data cube is composed of continuous two-dimensional images with certain spectral resolution intervals along the spectral axis.

Fig. 3 Schematic diagram of spectral cube

1. Simple classification of multispectral or hyperspectral

From the perspective of the acquisition methods of spectral information, hyperspectral imaging mainly includes the following two categories, as shown in Fig. 4:

(1) Based on the scanning method (multiple exposures), the method can also be divided into three forms: point scanning, line scanning and spectral scanning;

(2) Computational imaging method (hyperspectral imaging with single exposure);

Fig. 4 Schematic diagram of imaging method based on scanning method and calculation

1. Difference between hyperspectral and multispectral

In many cases, the change of reflectance characteristic spectrum of materials with respect to wavelength may be very complex, and other small features may not be resolved using coarser multispectral imaging methods.

Fig. 5 difference between multispectral and hyperspectral (3)

In the above figure, the indistinguishable substances are identified by multispectral imaging (left) and identified by hyperspectral imaging (right). The reason is that because hyperspectral has more spectral bands, more complex fingerprint features can be accurately obtained through higher spectral resolution.

In summary, hyperspectral has the following advantages over multispectral:

L obtain more complex and accurate spectral feature information through higher spectral resolution;

L obtain richer spectral band information and richer application scenarios;

L a set of hardware can be used to flexibly select spectral sequences according to applications, eliminating the process of system redesign;

1. Key parameters of hyperspectral imaging technology

For how to evaluate and understand a variety of spectral imaging technologies, we can focus on the main parameters such as spatial resolution, spectral resolution and accuracy, as well as the design complexity, physical size, system cost, mass production and reliability of the imaging system. In general, it is the core of the whole technology to obtain a stable, reliable and high-quality spectral image. Here are some key parameters.

Spatial resolution

Spatial resolution is one of the important indicators to evaluate the performance of sensors and images, and it is also used to characterize the spatial resolution by imaging pixels. Generally, the resolution is less than one million.

Spectral resolution

Refers to the number of bands selected by the spectral system, the wavelength position of each band, and the size of the wavelength interval. That is, the number of channels, the central wavelength and the bandwidth jointly determine the spectral resolution, which can be understood as the ability to distinguish or identify the light of each wavelength band in the spectrum. It is closely related to the resolution of the spectrometer. The finer the division, the more bands, and the higher the spectral resolution.

Spectral resolution is a concept. In a narrow sense, spectral resolution only refers to the band width. How to calculate it requires an index, and half wave width is this index.

Full width at half maximum

FWHM in English, also known as full width at half maximum, or half width at half maximum and half wave width. Refers to the spectral width at half of the spectral peak height. As shown in the following figure. Half wave width is a key technical parameter to measure hyperspectral imaging system, which can characterize its ability to subdivide and distinguish spectra. The common half wave width of multi spectrum is 10-1 λ Magnitude, hyperspectral is usually in the range of 10-2 λ magnitude.

Fig. 6 schematic diagram of fwhw

Spectral sampling rate

It is commonly understood as the number of channels, that is, the number of bands with different central wavelengths that can be obtained. There are usually 3-30 multispectral spectra and 100-300 hyperspectral spectra. Under the same conditions, the number of channels supported by the hyperspectral imaging system is determined by the fineness and flexibility of spectral adjustment.

reliability

After all, spectral information belongs to the information generated by the interaction between light and objects, which is relatively sensitive. Stability, consistency and anti vibration become very important. The drift of many devices with temperature change, mechanical change and time change is the main error source of the imaging system. The test accuracy in practical application greatly depends on the stability of the imaging system in this respect. It mainly focuses on temperature drift characteristics and vibration stability. Without stable and consistent physical data collection, it is very difficult to ensure that the algorithm based on data runs well.

1. Analysis of hyperspectral mainstream technology types

At present, hyperspectral imaging technology has developed rapidly, and the mainstream common ones include grating spectroscopy, acoustooptic tunable filter spectroscopy AOTF, liquid crystal optical filter LCTF, prism spectroscopy, chip coating, Fabry Perot cavity MEMS chips, etc. The basic principles and differences are briefly described below:

Grating splitting

After the one-dimensional information in space passes through the lens and the slit, the light of different wavelengths propagates according to different degrees of dispersion. Each point on the one-dimensional image is diffracted by the grating to form a spectral band, which is irradiated on the detector. The position and intensity of each pixel on the detector represent the spectrum and intensity. A point corresponds to a spectrum segment, and a line corresponds to a spectrum surface. Therefore, each imaging of the detector is the spectral information on a line in space. In order to obtain a two-dimensional image in space, the image and spectral data of the whole plane are acquired by mechanical scanning.

Prism splitting

After passing through the prism, the incident light is divided into different directions, and then irradiated on the detectors in different directions for imaging. After the prism splits light, filter films of different wavelengths are plated on the exit surface of the prism, so that detectors in different directions can collect different spectral information and realize simultaneous collection of spatial and spectral information.

Fig. 8 prism splitting schematic diagram 6

Since the system is based on a single discrete device, in order to ensure the spatial resolution and spectral resolution, optical devices such as objective lens, diaphragm, collimator and various lenses must be introduced, and the focusing and collimation between various devices must be considered, which leads to the high complexity, large volume, high cost and great limitation of the application range of the traditional system. For the modification of different application requirements, the system redesign complexity is very high. At present, most of these systems in the market are applied to scientific research and large-scale testing units.

Acousto optic tunable filter spectroscopic (AOTF)

AOTF is composed of acoustooptic medium, transducer and acoustic terminal. The RF driving signal excites the ultrasonic wave in the acoustooptic medium through the transducer. By changing the frequency of the RF drive signal, the wavelength of AOTF diffracted light can be changed, so as to realize the scanning of electrically tunable wavelength.

Figure 9 schematic diagram of AOTF 5

AOTF system consists of imaging objective lens + collimator lens + polarizer + Crystal + polarizer + objective lens + detector. In order to ensure that the incident light can be completely changed into parallel light after passing through the collimator lens, there are certain requirements for the field angle of the objective lens at the front end. The defect of this technology is that it can not be made into a large size. At present, what we can see is a single point spectrometer.

Liquid crystal optical filter (LCTF)

LCTF filter type spectral imaging technology is characterized by applying different voltages to adjust the phase difference caused by birefringent liquid crystal, so as to interfere with light of different wavelengths and realize continuous tunable scanning of different wavelengths. The basic structure is as follows:

Figure 10 LCTF structure figure 6

LCTF's liquid crystal is very sensitive to the external ambient temperature, which causes temperature drift and inaccurate detection results. Another disadvantage is that the cost is high and cannot be reduced. So far, from the research results and products that have been launched into the market, the technical route is not too optimistic.

Chip coating

The European Microelectronics Research Center IMEC has invested a lot of research in this area. It has developed a new hyperspectral imaging technology using a highly sensitive CCD chip and a SCMOS chip. Filter films of different wavelength bands are plated on the pixel of the detector to achieve hyperspectral imaging. As shown in the figure:

Fig. 11 mosaic coating diagram of 4x4 filter array splicing 6

This method of coating on the surface of CMOS brings a lot of low spatial resolution, requiring a high consistency of each coating, so the chip production process requires high technology, which poses a great challenge to mass production. Because the full spectrum can not be continuously adjusted, it is not flexible in application scenarios, and there will be certain restrictions, which can generally be qualitative.

Fabry Perot cavity MEMS chip

Fabry Perot interferometer (FPI or Fabry Perot cavity) is a multi beam interferometer composed of two parallel glass plates. The characteristic is that when the frequency of incident light meets its resonance condition, its transmission spectrum will have a high peak, corresponding to a high transmittance.

Figure 12 Fabry Perot interferometer

Hyperspectral imaging based on Fabry Perot cavity principle design, MEMS chip micromachining technology and mature image sensor technology can quickly achieve wide spectrum input, specific spectrum gated output, and complete image information acquisition of different spectra. This method is well combined with the existing device industry chain and module process, and has the advantages of small size and high cost performance of this MEMS device, which is suitable for mass production. Of course, this method is currently being studied by only individual research institutes and companies in the world, and the technical threshold is high.

Fig. 13 hyperspectral principle based on Fabry Perot cavity MEMS chip

1. Development trend and application prospect of hyperspectral imaging technology

The object component recognition of hyperspectral imaging technology has been proven in different applications, but the volume, cost and use experience of the existing technology have greatly limited its promotion and industrialization in more scene applications. Hyperspectral imaging technology must quickly reduce cost, miniaturize and miniaturize, and be more convenient in use experience.

Hyperspectral imaging technology, which was used in aerospace and military in the early days, has gradually stepped out of its application limitations. It has many applications in natural disaster prediction, environmental protection, biomedicine, agriculture, forestry, animal husbandry and fishery, and even many civil consumption and daily life scenarios have the possibility of using hyperspectral analysis technology more.

The application value of hyperspectral imaging technology has been proven in some non civilian fields, but there are still some difficulties that restrict its popularization into more fields, mainly including: 1) the hyperspectral camera system is complex and costly; 2) Large volume and heavy weight; 3) Whether the use experience is convenient. At present, manufacturers are trying to overcome these shortcomings. With the expansion of application demand, sensor design and technological innovation, and the mature development of the industrial chain, these are the main driving forces for the promotion of hyperspectral imaging. If hyperspectral cameras are introduced into daily production, such as smart phones and Internet of things devices, new technologies will bring richer applications and smarter functions, which will have a great impact on lifestyle, and really let hyperspectral cameras enter the homes of ordinary people.