Monitoring of Plankton Spatial and Temporal Characteristics With the Use of a Submersible Digital Holographic Camera

Introduction

The main objectives of environmental monitoring of any water area include regular analysis of environmental conditions and forecasting of ecosystem changes under the influence of technogenic and other factors. Such analysis is performed against the background of the natural variability of the water area conditions, daily and seasonal temperature fluctuations, illumination, etc. The conservation of biodiversity typical for this water area, especially for pelagic plankton, may indicate the habitat optimum factors.

The pessimal factor associated with anthropogenic activities may change the number of different plankton species due to the difference in their environmental tolerance and vulnerability to anthropogenic effects. Thus, the violation of alpha diversity parameters, such as the species richness and the evenness of species abundance, indicates some environmental hazards.

Hydrobiological research that traditionally involves the sampling of phytoplankton, zooplankton, zoobenthos, periphyton, and field ichthyological survey forms a crucial part of environmental monitoring of aquatic environments (Chiba et al., 2018; Batten et al., 2019; Lombard et al., 2019). For instance, such research is mandatory in the survey of the territory of future construction and monitoring of water areas within hazardous facilities. Furthermore, field and laboratory works involve the following:

– study of species diversity and spatial and temporal organization of zooplankton,

– study of seasonal dynamics of species diversity and abundance indicators,

– determination of the ecosystem trophic status according to the ratio of eutrophic and oligotrophic zooplankton species (Hakkari, 1972). The standard monitoring includes water sampling and subsequent laboratory analysis of plankton content. This process is time-consuming and cannot be frequently repeated since the sampling itself perturbs the controlled volume.

Digital holography is a promising method for the in situ study of plankton (Sun et al., 2008; Watson, 2011; Talapatra et al., 2012; Bochdansky et al., 2013; Rotermund et al., 2016; Davies and Nepstad, 2017; Guo et al., 2018; Lombard et al., 2019).

The study describes the possibilities and the peculiarities of using a submersible digital holographic camera (DHC) (Dyomin et al., 2019d) for such monitoring tasks as part of the hardware and the software complex developed by the research team. The main advantage of using DHC is the possibility of studying plankton in its habitat without sampling and interference into the biocenosis. DHC is also characterized by a fairly representative medium volume recorded per exposure – up to 1 L, while in similar holographic cameras the analyzed medium volume does not exceed 40 cm³ (Watson, 2011; Talapatra et al., 2012; Rotermund et al., 2016; Dyomin et al., 2019b; HoloSea, 2020; Lisst-Holo2, 2020). Unlike photographic methods of particle research (Olson and Sosik, 2007; Cowen and Guigand, 2008; Ohman et al., 2012), it is possible when reconstructing a digital hologram to obtain sharp images of all particles that were in a given medium volume during exposure. In addition to the digital holographic camera, the hardware and the software complex includes hydrophysical sensors (pressure gauge, temperature gauge, and microwave conductivity sensor) and an onboard computer. All components are integrated into the local network controlled by the onboard computer, and the communication channel ensures continuous real-time data transfer to a shipboard computer. DHC records digital holograms of the controlled medium volume in a given sequence. The shipboard computer performs the numerical processing of holograms and automatically extracts information on plankton from them. Unlike other plankton classification algorithms (Lumini and Nanni, 2015; Lombard et al., 2019), the proposed automatic classification algorithm using the morphological parameter provides for fast (<1 s) classification of plankton according to the chosen taxa. The automation of information acquisition provides monitoring research of species diversity and spatial and temporal organization of zooplankton in conjunction with hydrophysical parameters of the medium.

Methods

The developed DHC used in this study (Dyomin et al., 2018a,b,c,d, 2019a,b,c,d) provides for an automatic registration of a digital hologram that contains information on the entire studied volume with plankton. DHC ensures the detailed study of particles ranging from 200 μm in size. The application of such camera is described in Dyomin et al. (2018a). The DHC optical scheme is shown in Figure 1A. The light from the laser source (4) passes through the collimator (5), then through the volume with the analyzed particles (3), reflecting on a mirror prism system, and enters the camera (6) through the receiving lens. Thus, the camera (6) registers a digital hologram of the analyzed volume with the particles representing an interference pattern of two coherent waves – scattered on particles and passed without scattering. The design and the specifications of the camera are described in more detail in our previous studies (Dyomin et al., 2019a, c). Figure 1C shows the DHC layout.

Figure 1. (A) Digital hologram recording scheme using submersible digital holographic camera (DHC) (folded configuration) in laboratory conditions. (B) Digital hologram recording scheme using DHC (laboratory linear configuration) in laboratory conditions. 1 – DHC, 2 – laboratory water tank (A) or cuvette (B) with plankton, 3 – studied (recorded) volume, 4 – laser, 5 – beam expander, 6 – CMOS camera. (C) DHC horizontal layout as part of a hydrobiological probe in the study of vertical plankton distributions of Lake Baikal in February 2019 and DHC vertical layout for the study of plankton temporal characteristics.

The algorithm of numerical particle image restoration from the digital hologram is well known (Schnars and Juptner, 2002; Watson, 2018) and underlies the base software. The numerical processing of digital holograms and the algorithm of parameter determination and automatic classification of particles are described in our previous works (Dyomin and Olshukov, 2016; Dyomin et al., 2017; Dyomin et al., 2018b) and form the problem-oriented software. The entire software is commercial, so let us list its main components without describing them in detail.

The sequence of operations performed using DHC and registered digital holograms is called the DHC technology. In addition to digital hologram recording, the DHC technology includes a set of numerical operations on its special processing – hologram preprocessing to eliminate edge effects and to level the background noise (Dyomin and Olshukov, 2016; Dyomin et al., 2018b), restoration of holographic images of cross-sections of medium volume at various distances (Schnars and Juptner, 2002; Watson, 2018), and a 2D display of holographic volume with plankton (Dyomin et al., 2018b). The 2D display thus constructed consists of focused images of all plankton species in the studied medium volume at the time of hologram registration (Dyomin et al., 2018b). Besides that, the 3D coordinates of each species are known: the longitudinal coordinate is determined from the position of the best focus plane of a restored particle image; the coordinates of the image center of gravity are taken as transverse particle coordinates (Dyomin and Kamenev, 2016a, b).

This is followed by automatic binarization of the 2D display, identification of the image of each plankton species, and definition of its characteristics according to the parameters of a rectangle circumscribed around its restored image: size (width and length of a rectangle), tilt angle, boundary length, and area. Simultaneously, for each image of plankton species, we define the compactness (ratio of the boundary length square to the area of the image), morphological parameter M (ratio of the width and length H of the circumscribed rectangle), and the presence of antennas or other extremities (Dyomin et al., 2018b).

These parameters are used for the automatic classification of planktonic particles according to chosen taxonomic groups. The decision tree for automatic classification (Figure 2A) includes eight taxa: Chaetognatha, Copepoda, Appendicularia, Cladocera, Rotifera, colonies of phytoplankton, phytoplankton chains, and others (jellyfishes, meroplankton, and others) (Dyomin et al., 2017, 2018b).

Figure 2. (A) Decision tree of automatic classification of plankton individuals, where M – morphological parameter and H – rectangle length. 2D display of holographic image of the studied volume with (B) Daphnia magna and (C) Epischura baikalensis individuals (nauplii and adult specimen), where the result of automatic classification is marked with the color of the rectangle circumscribed around the particle: black – phytoplankton colonies, dark blue – Copepoda, blue – Cladocera, and red – other.

The preliminary-formed database on plankton of this water area allows obtaining the species composition of particles (by orders) and the species distribution of particles by size as well as the species concentration of particles in the analyzed volume during monitoring.

Within the scope of the present work, the laboratory and the field studies of plankton were carried out using DHC and DHC technology to test and verify the features of the automatic classification of plankton as well as to confirm the possibilities of monitoring its temporal and spatial characteristics. The described results demonstrate only some capabilities of the developed equipment and software and make no pretense to comprehensive information on plankton species. The problem-oriented part of the software can be expanded or narrowed depending on the current task.

Results and Discussion

Study of the Accuracy of Plankton Automatic Classification Algorithm

The laboratory studies using DHC were conducted to check the accuracy of the automatic classification algorithm. We used the camera that makes it possible to record a hologram with a water volume of 0.6 dm³ per exposure. The camera was placed in a 90 dm³ laboratory water tank filled with culture water. The entire optical part of the camera was submerged into the water (Figure 1A). We used Daphnia pulex and Cyclops strenuus sampled from a natural population in a freshwater reservoir near Tomsk. One hundred C. strenuus individuals were placed in the water tank, and then DHC was started in a mode of digital hologram recording every 4 s, with a total of 100 holograms recorded. Then, 100 C. strenuus individuals were additionally placed in the water tank and 100 holograms were recorded in the same mode. Furthermore, we continued adding individuals and registering digital holograms in the following order: 100 C. strenuus individuals, 100 D. pulex individuals, 100 D. pulex individuals, and 100 D. pulex individuals. During the last registration of digital holograms, there were 300 C. strenuus individuals and 300 D. pulex individuals. Thus, a total of 600 digital holograms were recorded throughout the experiment. The DHC software made it possible to obtain a database of particles recognized in the investigated volume from restored holographic images as well as to assess the parameters of their automatic classification algorithm. The classification was carried out using the decision tree (Figure 2A).

Supplementary Table S1 presents the confusion matrices compiled from laboratory studies of the automatic classification accuracy (Olson and Delen, 2008). The classification accuracy is understood here as the ratio of the number of accurately classified individuals to the total number of identified planktonic individuals of all taxa (the accuracy is indicated in the first cell for each confusion matrix in Supplementary Table S1).

The matrices summarize data on each group of 100 processed digital holograms. The rows of confusion matrices denote true taxa, while the columns define the classifier solutions (predictions). The confusion matrix is built by counting the number of times a classifier has identified a planktonic individual of a true taxon as a certain taxon. For example, for the case of recording 100 C. strenuus individuals in Supplementary Table S1, 22 Copepoda individuals were wrongly classified as Cladocera class, whereas 79 individuals were classified correctly as Copepoda taxon (diagonal element of the confusion matrix). The validity of automatic classification was verified manually by an operator. The precision and the recall (Olson and Delen, 2008) for each class are specified in the confusion matrices: precision equals the ratio of the corresponding diagonal element of the matrix to the sum of the entire column of the class; recall is the ratio of the diagonal element of the matrix to the sum of the entire row of the class. Precision determines the proportion of planktonic individuals classified as a certain taxon and indeed related to a given taxon (precision demonstrates the classifier’s ability to distinguish a given taxon from others). Recall shows the share of plankton individuals of a certain taxon among all individuals of a given taxon found by the classifier (recall demonstrates the ability of the algorithm to detect a given taxon in general). The confusion matrices make it possible to visualize the result of the classification algorithm and to evaluate its quality numerically for each specific water area.

The laboratory studies with D. pulex (Cladocera taxon) and C. strenuus (Copepoda taxon) showed that the accuracy of the automatic classification of plankton was 61 ± 3%. The confusion matrices demonstrate that the classification accuracy for Copepoda taxon was 86 ± 9%, for Cladocera 77 ± 2%, and for others 76 ± 15%. The recall of the classification for the Copepoda taxon was 73 ± 7%, for Cladocera 52 ± 19%, and for others 45 ± 10%. The averaging was done for all relevant values from the confusion matrices (i.e., for 100 × 6 digital holograms) and given above in mean ± SD format. A low recall of Cladocera and others indicates a low automatic classification ability to detect these taxa. This is caused by a coherent noise in the recovered holographic images that make it difficult to detect the presence of antennas. It shall be noted that the classification accuracy in Supplementary Table S1 for particles classified as phytoplankton colony and other is caused not only by the errors of the algorithm but also by the presence of feeds and by-products of plankton.

Examples of Application of DHC Technology and Automatic Classification Algorithm in Laboratory Experiments

Laboratory testing was carried out using a digital holographic camera model, the scheme of which is shown in Figure 1B. We used Daphnia magna sampled from a natural population in a freshwater reservoir near Tomsk. A cuvette filled with water containing plankton was placed between the camera and the laser (Figure 1B). The recorded volume was 0.11 dm³. Figure 2B shows the example of an image of D. magna individuals numerically reconstructed from a digital hologram and a 2D display of the studied volume with focused images of individuals. The figure also shows rectangles circumscribed around the particle images, with the color of the rectangle corresponding to a given kind of particles and planktonic individuals. Supplementary Figure S1 shows the histograms of particle distribution by automatic classification taxa and by size. Here and on the following histograms, the particle size is understood as the length of a circumscribed rectangle. The classification time was about 0.9 s, with 45 particles detected in the recovered image and the frame size of the processed image amounting to 2,048 × 2,048 pixels. Hereinafter, the classification time is given without regard to the recovery time of the digital holograms, construction of the two-dimensional display of the holographic volume, and detection of particles. The classification accuracy was 63%. Out of nine D. magna individuals present in the frame, two are wrongly classified as Copepoda. Besides that, there are feeds and by-products of the culture in the aquarium, which the algorithm classifies as other and colony of phytoplankton. In turn, the last solution of the algorithm is erroneous and affects the accuracy of classification.

Figure 2C and Supplementary Figure S2 demonstrate the performance of the DHC technology and automatic classification using another laboratory experiment – processing of digital holograms of a plankton sample obtained via the traditional Juday net in Lake Baikal in 2008. This sample in a cuvette was placed in a digital holographic camera model, as shown in Figure 1B (cuvette 3). The recorded volume amounted to 0.01 dm³. The classification accuracy of Epischura baikalensis (Copepoda taxon) was 56%. The classification time also depends on the frame size of a processed image and on the number of particles in a frame. Thus, unlike the previous case, in this example, the frame size of the processed image was 1,024 × 1,024 pixels, so the classification time is less and only takes about 0.5 s, with 60 particles detected in the recovered image.

Examples of Application of DHC Technology and Automatic Classification Algorithm in Full-Scale Experiments

The next example of the DHC technology and classification algorithm involves DHC testing during the summer Arctic mission of the Institute of Oceanology RAS on the research ship Academician Mstislav Keldysh in the Kara Sea in 2016 (Dyomin et al., 2018a). During the mission, the DHC was submerged to a given depth on a winch to record digital holograms. The registered digital holograms were transmitted to a shipboard computer for further processing. The recorded volume per exposure was 1 dm³. Supplementary Figure S3 shows an example of the images of planktonic particles recovered through processing of one of the holograms recorded in situ during the mission in the Kara Sea. Supplementary Figure S4 shows the histograms of particle distribution by automatic classification taxa and size. The classification accuracy is about 73%. The classification time is about 0.1 s, with the frame size of the processed image being 1,024 × 1,024 pixels and 16 particles were detected in the image.

Field studies of vertical distributions of various types of plankton were carried out in the Blue Bay of the Black Sea in 2019. The measurements were performed from the board of the research ship Ashamba. DHC, as part of the hardware and the software complex, was submerged via an onboard winch to a depth of up to 7 m. The sinking speed of the device was about 0.1 m/s, with data being recorded every second. Temperature and pressure were recorded simultaneously with the digital holograms of plankton. Figure 3 shows the vertical distribution profile of plankton concentration by taxa (spp/m³ is referred to individuals/m³). As a result of automatic classification, no Chaetognatha, Appendicularia, Rotifera, and phytoplankton chains were found in the reconstructed holographic images, so they are not illustrated on the scheme. The time of obtaining one reference point of the scheme on the shipboard computer did not exceed 20 s; the full background profile to a depth of 7 m was formed within 2 min.

Figure 3. Distribution profile of plankton concentrations of various taxa in depth up to 6 m. The following taxa are present: Copepoda (blue), Cladocera (turquoise), other (red), and phytoplankton colony (black).

Discussion of Experimental Results on Plankton Classification Using the DHC Technology

The experiments showed that, within the accomplished version, the classification algorithm using the morphological parameter makes it possible to solve the problem quickly (the time required to obtain the result is <1 s and depends on the number of plankton particles and the frame size of the restored image); however, the classification accuracy by orders varies within 50–60%. A relatively low accuracy is bound to both objective reasons (by-products, turbulence, and, as a result, noise of the restored particle images) and the software attempting to carry out a rapid preliminary classification. An increase in accuracy will require increasing the time for processing and recognition of images, i.e., depending on the task, here it is necessary to observe the “accuracy-computing speed” balance. Besides that, in order to classify such taxa as the colonies of phytoplankton and other with higher accuracy in the future, it is necessary to supplement the capabilities of hardware and problem-oriented software, in particular, to introduce the diagnostics of additional features such as “living” or “not living” and “phytoplankton” or “zooplankton.”

Experimental Marine Studies of Plankton Temporal Characteristics

One of the experiments to study the temporal characteristics of plankton was conducted in the Kola Bay. The data were obtained in 2018, on the research ship Dalnie Zelentsy during its anchorage, using DHC. DHC as part of a hardware and a software complex including hydrophysical sensors (pressure gauge and microwave conductivity sensor) was fixed at a depth of 1.5 m from the bottom (6.5–8.5 m from the water surface, depending on the tidal processes). The conductivity and the pressure details were transmitted from the sensors of the hardware and the software complex to the shipboard computer simultaneously with the digital holograms of plankton. The data on plankton were acquired via the shipboard computer from digital holograms using DHC technology. One of the features of this experiment was the remote operation of the device – the device was connected and controlled remotely from Tomsk; simultaneously, the data (digital holograms and data on plankton, pressure, and conductivity) were transmitted via communication lines to Tomsk.

In order to obtain data on plankton concentration, the measurements were performed during the day with the frequency of one count per hour. The volume recorded per exposure was 1 dm³. Figure 4A shows the daily range of the total concentration of all particles identified in images recovered from digital holograms as well as pressure and conductivity. Severe daily variations in the water level on the day of the measurements, caused by the tidal processes in the Kola Bay, resulted in significant changes in the depth of the coastal area from 8 to 10 m. This, in turn, led to changes in hydrostatic pressure in the anchorage area. The daily variations of coastal plankton concentration also strongly changed and were 20,000–220,000 spp/m³. The given data (Figure 4A) show that the change in plankton concentration follows the tidal processes, with a delay of about 4 h. It can reasonably be assumed that this is caused by the horizontal advection of plankton toward the tidal movement of water masses since the device is fixed at a constant distance from the bottom (1.5 m). The results clearly illustrate the capabilities of the DHC technology; however, in this case, they are not sufficient for full interpretation, in particular, to explain the 4-h delay in the change in plankton concentration relative to the ebb and the flow.

Figure 4. Data obtained with the use of the digital holographic camera as part of the hardware and the software complex (A) in the Kola Bay at the stationary station, at a fixed depth of 1.5 m from the bottom (25.06.2018) and (B) in the Black Sea in the area of the stationary hydrophysical station (28.08.2018). The solid lines indicate (A) 6th- and (B) 4th-degree polynomial approximations.

Another type of plankton migration, vertical, is illustrated by the dependencies shown in Figure 4B, obtained during the mission to the Black Sea, near the Katsiveli village.

During the mission to the Black Sea, the DHC, as part of the hardware and the software complex, was operating from a stationary hydrophysical platform. The volume recorded per exposure, as in the previous case, was 1 dm³. In order to study the temporal characteristics of plankton, the data were recorded every hour during the day at a depth of 10 m, fixed relative to the water surface, unlike in the previous case when the distance from the bottom was fixed. Conductivity and temperature details were transmitted from the sensors of the hardware and the software complex to the shipboard computer simultaneously with the digital holograms of plankton. The obtained results are graphically shown in Figure 4B. The concentration of plankton in the Black Sea is significantly higher than that in the waters of the Kola Bay in absolute magnitude, but the dynamics of these changes is slightly smaller. The daily variations of plankton concentration were 200,000–700,000 spp/m³. It shall be noted that the daily variation of plankton concentration correlates reasonably well with the temperature change, thus indicating a vertical migration of plankton. At the same time, its noticeable deviation in pre-morning hours, which is most likely associated with sunrise, indicates the multifactority of processes.

Thus, the results given in this section demonstrate quite good DHC ability with respect to the study of plankton in a wide range of concentrations and in different weather conditions. Besides that, these clearly illustrate the need to register hydrophysical parameters simultaneously with plankton holographing to ensure a reliable interpretation of the processes.

Conclusion

These experimental results show that a submersible digital holographic camera based on the DHC technology as part of a hardware and a software complex allows carrying out in situ measurements of plankton, automating the process of obtaining data on its temporal and spatial distribution as well as classifying plankton species by classes within the specified taxonomic groups (Chaetognatha, Copepoda, Appendicularia, Cladocera, Rotifera, phytoplankton colonies, phytoplankton chains, and others).

The study presents the results of automatic classification of plankton in laboratory and field conditions together with depth profiles and daily variations of plankton concentration as measured by a submersible digital holographic camera in natural conditions (Kara Sea, Kola Bay, and Black Sea). The obtained results demonstrate quite good DHC ability with respect to the study of plankton in a wide range of concentrations and in different weather conditions. Besides that, these clearly illustrate the need to register hydrophysical parameters simultaneously with plankton holographing to ensure a reliable interpretation of the processes.

It is shown that, within the accomplished version, the classification algorithm using the morphological parameter makes it possible to solve the problem quickly (the time required to obtain the result is <1 s and depends on the number of plankton particles and the frame size of a restored image); however, the classification accuracy by orders varies within 50–60%.

The accuracy increase will require increasing the time for processing and recognition of images, i.e., depending on the task, here it is necessary to observe the “accuracy-computing speed” balance. Besides that, in order to classify such taxa as the colonies of phytoplankton and other with higher accuracy in the future, it is necessary to supplement the capabilities of hardware and problem-oriented software, in particular, to introduce the diagnostics of additional features such as “living” or “not living” and “phytoplankton” or “zooplankton.”

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Author Contributions

VD and IP contributed to the concept and the design of the study and the results interpretation. AD developed and organized the soft database and performed the statistical analysis. SM performed the laboratory experiments and the validation of classification. NK, AO, and SD performed the field experiments. AD and IP wrote the first draft of the manuscript. VD contributed to revisions and corrections. All authors reviewed and approved the final version of the manuscript.

Funding

The research using DHC in laboratory and full-scale conditions was supported by the Tomsk State University Competitiveness Improvement Program under Academician D.I. Mendeleev Fund Program. The development of algorithms for particles classification based on the information extracted from digital holograms was funded by RFBR, project number 19-32-90233.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We express our gratitude to the employees of the Center for Biosafety of Nanomaterials and Nanotechnologies of TSU for the laboratory experiment on the verification of classification capabilities and for the useful discussion.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmars.2020.00653/full#supplementary-material

FIGURE S1 | (A) Histogram of particle distribution by taxon (automatic classification). Taxon designation: 1 – phytoplankton colony, 2 – phytoplankton chain, 3 – Copepoda, 4 – Appendicularia, 5 – Rotifera, 6 – Cladocera, 7 – other. (B) Histogram of particle size distribution.

FIGURE S2 | (A) Histogram of particle distribution by taxon (automatic classification). Taxon designation: 1 – phytoplankton colony, 2 – phytoplankton chain, 3 – Copepoda, 4 – Appendicularia, 5 – Rotifera, 6 – Cladocera, 7 – other. (B) Histogram of particle size distribution.

FIGURE S3 | 2D display of holographic image obtained using digital holographic camera in the water area of the Kara Sea. The result of automatic classification – the color of the rectangle circumscribed around the particle: black, phytoplankton colony; gray, phytoplankton chain; dark blue, Copepoda; blue, Cladocera; red, other.

FIGURE S4 | (A) Histogram of particle distribution by taxon (automatic classification). Taxon designation: 1 – phytoplankton colony, 2 – phytoplankton chain, 3 – Copepoda, 4 – Appendicularia, 5 – Rotifera, 6 – Cladocera, 7 – other. (B) Histogram of particle size distribution.

TABLE S1 | Confusion matrices.

References