Investigation of Chromatic Aberration and Its Influence on the Processing of Underwater Imagery

Abstract

The number of researchers utilising imagery for the 3D reconstruction of underwater natural (e.g., reefs) and man-made structures (e.g., shipwrecks) is increasing. Often, the same procedures and software solutions are used for processing the images as in-air without considering additional aberrations that can be caused by the change of the medium from air to water. For instance, several publications mention the presence of chromatic aberration (CA). The aim of this paper is to investigate CA effects in low-cost camera systems (several GoPro cameras) operated in an underwater environment. We found that underwater and in-air distortion profiles differed by more than 1000 times in terms of maximum displacement and in terms of curvature. Moreover, significant CA effects were found in the underwater profiles that did not exist in-air. Furthermore, the paper investigates the effect of adjustment constraints imposed on the underwater self-calibration and the reliability of the interior orientation parameters. The analysis of the precision shows that in-air RMS values are just due to random errors. In contrast, the underwater calibration RMS values are 3x-6x higher than the exterior orientation parameter (EOP) precision, so these values contain both random error and the systematic effects from the CA. The accuracy assessment shows significant differences.

1. Introduction

Low- and medium-cost camera solutions are used for a wide range of underwater applications. Many of those applications focus on coral reefs including habitat mapping [1], dimensional and colorimetric reconstruction [2] and fractal dimension and vector dispersion estimation [3]. Other fields are the creation of 3D reconstructions and virtual reality models (VRMs) for archaeological and touristic sites [4]. The photogrammetric processing of the images is often the first step followed by the deriving object and discipline-specific attributes [5].

The basis for the photogrammetric processing of any images (captured underwater or in-air) is usually predicated on the assumption that light rays travel in a straight line. This allows the application of the collinearity equations. Any deviation in the direction of the light ray can be modeled by introducing additional camera parameters.

In general, image imperfections are called aberrations. Next to spherical aberration, coma, astigmatism, curvature of field and distortion, there are also chromatic aberrations (CA) in colour images [6]. In conventional photogrammetry, with normal lenses and in-air operation, the main distortions are radial lens distortion, decentering lens distortion and linear distortions. Due to their magnitude and their systematic influence, the collinearity equations are extended to model and correct the distortions. The distortion model parameters are estimated by applying camera self-calibration methods and procedures.

Camera calibration methods and procedures are well established [7,8]. The most commonly used method is target field calibration in which multiple, convergent images of a fixed calibration frame are captured to determine the parameters of the camera in the medium in which the camera is to be used. To determine the internal geometric characteristics of the camera, a parameter set is used comprising the principal distance, principal point location, radial [9] and decentering [10] lens distortions, plus affinity and non-orthogonality terms to compensate for electronic or minor optical effects [8]. These parameters are often referred to as interior orientation parameters (IOPs) and can be determined together with the exterior orientation parameters (EOPs) using a least-squares adjustment. The EOPs comprise the position of the perspective centre and three rotation angles per image. This approach is very flexible in that it permits the inclusion of several different constraints into the adjustment.

There are two types of CA: longitudinal and lateral or transverse [6]. Longitudinal aberration is measured in the direction parallel to the optical axis. The variation of the principal distance is caused because short wavelengths (i.e., blue) are more strongly refracted by a positive lens and hence, come into focus closer to the lens. On the other hand, a longer wavelength (red) focused further from the lens (Figure 1, left). Lateral aberration is measured perpendicular to the optical axis and is caused by the dispersion of light by the lens system. Therefore, lateral aberrations are also called chromatic differences of magnification or colour misalignment (Figure 1, right). Most lenses are designed to capture images in-air and the effects of chromatic aberrations are minimised using achromatic elements in the lens design [6].

CA can be modeled by different principal distances and radial lens distortion parameters for each colour using the camera calibration method based on [8,9]. This is the method which will be used in this paper. However, it must be mentioned that the modelling and correction of CA is approached differently in various fields. For example, the correction of CA in dermoscopes is considered using a second order radial distortion model of artificial white centroids placed in the field of view of the camera and extracted from the image [12], video processing using an aberration-tolerant demosaicking algorithm [13], color plane warping [14] and light propagation modeling using stereo-camera systems [15].

Several researchers have pointed out that the camera parameters and the accuracy that can be derived processing underwater images can be different from the parameters and accuracy that can be derived by processing images captured in the air. However, while the processing of images underwater can be more challenging [16], it does not mean that the accuracy which can be achieved has to be worse [17].

As mentioned, due to the design of the lens system, there is usually no need to apply colour corrections and chromatic aberrations for short distances in the environment for which the camera system/lenses were designed. Camera systems are usually optimised for in-air operation. The change from air to underwater and, therefore, the different absorption and refraction effects of water may require the introduction of additional correction or modelling if no further considerations are made about the distortion characteristics of the specific camera in combination with the used port (flat or dome port) [18].

In [19], the need to correct or model refraction effects in underwater environments is discussed, pointing out that by far the most common approach is to correct the refraction effect using absorption by the physical calibration parameters. The primary effect can be absorbed by the radial lens distortion parameters. Small, asymmetric effects caused by alignment errors between the optical axis and the housing port (secondary effects) may also exist and can be absorbed as decentering lens distortion parameters.

The aim of this paper was to investigate chromatic aberration effects in low-cost camera systems (several GoPro cameras) calibrated and operated in an underwater environment. The paper will compare the camera parameters of the same cameras operated in-air and underwater as well as investigate the effect of adjustment constraints to the underwater processed images. The reliability and the accuracy of the photogrammetric processed underwater images will be also investigated.

The paper is structured as follows. In the next section, publications that focus on the comparison of camera calibration parameters for in-air and underwater operation and CA will be reviewed. Then, the methodology used for data processing is introduced followed by the introduction of the datasets used in this paper. Then, the results of the different comparison are presented and discussed. The paper will close with a conclusion.

2. Related Work

The number of publications focusing on the direct comparison of camera parameters and performance for in-air and underwater applications is limited. One example of validating the performance of a camera system in both environments has been published by [17]. The focus of this research was on image-based motion analysis as a function of different camera setups and camera settings including the acquisition frequency, the image resolution and the field of view. The authors used GoProHero3+ operated in-air (laboratory) and underwater (swimming pool) to evaluate the three-dimensional (3D) accuracy achievable for the image-based motion analysis. Statistically, the 3D accuracy obtained in-air was poorer than the accuracy achieved in the underwater environment across all the tested camera configurations. However, a conclusion of how the camera parameters are affected by the change of the environment from in-air to underwater was not provided.

In [20], it is reported that the very high accuracy and precision using GoPros underwater is possible. Using multiple GoPro cameras in different test sets, the paper further validated that the consistency of the interior orientation parameters is stable. However, the paper did not investigate CA or provide a comparison of the results achievable underwater and in air.

In contrast, several other authors include CA in their investigations. For instance, [21,22] report CA in fisheye lens systems. In [22], the focus is on the in-air calibration of two lenses and it is interesting to note that this paper found that the aperture affects the magnitude of the CA. However, a systematic influence between the two different lens systems on CA in the different colour bands could not be observed.

In [23], the presence of CA in underwater imagery is reported, which was captured with a camera system coupled with a planar housing port. This paper further investigated the accuracy assessment of dome and planar housings on a distance observation. The dome housing produced results fifty times better than the those from the planar housing.

The authors in [11,23] report CA in underwater imagery. Significant corrections for the radial and decentering distortion parameters indicating lateral CA were observed in images taken from a GoPro Hero Black 3 [11]. A significant trend of different principal distances’ indication longitudinal CA was not observed. In [11], the authors reported in addition that the found lateral CA did not confirm the theory presented in Figure 1. Possible reasons were that the calibration frame only covered 1/3 of the camera’s field of view.

At least two options exist for mitigating errors due to CA [24]: modelling the effect with colour-dependent calibration parameters and correction by image pre-processing. One approach for modelling CA is based on the self-calibrating bundle adjustment [25]. In [26], the self-calibration approach is investigated using a bundle adjustment for cameras of a Ladybug system in-air as well as several GoPro cameras underwater. In contrast to previous works, the application of different constraints was specifically investigated. Overall, four different adjustments per camera were performed, and their results were evaluated especially focusing on CA. The first adjustment did not introduce any constraints, so all colour bands were adjusted independently. The second adjustment considered the constraint that the points of the calibration field have the same coordinates in object space. Building on the second adjustment, the third adjustment added constraints to force the red, green and blue exterior orientation parameters to be equal. The final adjustment used the constraint of the second adjustment as well as the constraints to enforce the equivalence of the principal point coordinates of the cameras corresponding to the three spectral bands. The third adjustment used by [26] is comparable to the adjustment used by [25,26], who concluded that ether common object points or alternatively, common object points with EOP constraints are needed.

Modelling approaches like those of [27] only report limited results. The main conclusion of this paper was that the CA can be expressed by a cubic function of the radial distance from an image frame center. This method corrects CA satisfactorily enough in many cases.

In [28], an underwater measurement of coral reefs with GoPro cameras was performed. Instead of using a bundle adjustment approach in which individual colour images are calibrated, a relative CA correction was performed: two channels are corrected relative to the third. More specifically, the red and blue channels were aligned to the green channel. In a first step, the displacements between the bands is estimated using optical flow. Then, the parameters modelling the displacement are estimated. For this step, two models are investigated—the Brown model using three radial distortion parameters, two decentering distortion parameters and the principal point offset, and a least-squares collocation method. After the displacement parameters are approximated, the blue and red channels are corrected and shifted to the green channel. The deterministic part of the CA is modeled as a second-order polynomial. Though not stated, the collocation approach assumes that the CA is a stationary random process. Furthermore, Gaussian covariance function is used. The paper concludes the collocation performed better overall.

To summarise, existing papers compared the in-air and underwater performances of the same camera but did not analyse the CA. Publications related to CA used different cameras for underwater and in-air calibration and were mainly focused on ether modelling the shift between different bands or investigating in different adjustment models and their constraints. Other aspects covered are the dependencies of the CA on aperture and the comparison of accuracy using different housings.

The contributions of this paper include the comparison of in-air and underwater camera calibration parameters of the same cameras with a specific focus on quantifying any CA. Furthermore, the effectiveness of the various constraints on the EOP is investigated for both in-air and underwater images captured by the same cameras. Moreover, the repeatability of the calculated camera calibration parameters in the underwater environment is investigated. Finally, an experiment designed to replicate a real-world, underwater application using imagery is reported. The effect of CA is analysed by quantifying 3D reconstruction accuracy in terms of spatial distances.

3. Methodology

In this paper, a similar method to [25,26] is used. In this section, the functional model and the add adjustment constraints will be introduced.

3.1. Functional Model

As outlined in the Introduction, the collinearity condition is assumed, i.e., an object point i (X, Y, Z), its homologous image point (x, y)_i in image j and the perspective centre of image j (X^c, Y^c, Z^c) lie on a straight line in a three-dimensional space. All light rays entering the lens system form a bundle passing through the perspective centre. In this paper, images are split into separate red (R), green (G) and blue (B) images. Independent image point measurements are made in each spectral band image. For each spectral band k ϵ {B, G, R}, the collinearity equations are given by

$$x_{ijk}+{\widehat{v}}_{x_{ijk}}=x_{p_{jk}}-c_{jk}\frac{{\mathrm{U}}_{\mathrm{ij}}}{{\mathrm{W}}_{\mathrm{ij}}}+\Delta x_{ijk}$$

(1)

$$y_{ijk}+{\widehat{v}}_{y_{ijk}}=y_{p_{jk}}-c_{jk}\frac{{\mathrm{V}}_{\mathrm{ij}}}{{\mathrm{W}}_{\mathrm{ij}}}+\Delta y_{ijk}$$

(2)

where ${\widehat{v}}_x$, ${\widehat{v}}_y$ are the image point residuals, x_p, y_p are the coordinates of the principal point offset, c is the principal distance and Δx, Δy are the additive correction terms. Furthermore, (U, V, W) are formulated as

$$\left(\begin{array}{c}U\\ V\\ W\end{array}\right)=M \left({\left(\begin{array}{c}X\\ Y\\ Z\end{array}\right)}_i-{\left(\begin{array}{c}X\\ Y\\ Z\end{array}\right)}_j^C\right)$$

(3)

where M is the matrix used for the rotation from object space to image space that can be parameterised as

$$M=R_3\left({\kappa }_j\right)R_2\left({\phi }_j\right)R_1\left({\omega }_j\right)$$

(4)

and ω, ϕ and κ, are the rotation angles around the X, Y and Z axes, respectively. The (Δx, Δy) terms are corrections required to account for non-collinearity due to lens distortion, including CA.

Radial lens distortion is a systematic error source inherent to all lens systems. Out of all systematic errors, the radial lens distortion usually has the largest magnitude, which leads to symmetric image point displacement around the principal point. This displacement can be either towards the principal point (barrel distortion) or away from the principal point (pincushion distortion). In air, wide-angle and fisheye lenses usually have a very strong barrel distortion leading to a very large displacement of points.

In this paper, the functional model for the radial lens distortion is an odd-powered polynomial that models the radial displacement Δr_k as follows:

$$\Delta r_k=k_{1k}{r}^3+k_{2k}{r}^5+k_{3k}{r}^7+k_{4k}{r}^9+k_{5k}{r}^{11}$$

(5)

where r is the distance of the image point from the principal point. In general, five parameters were used to model the radial lens distortions for the in-air calibration. Though the number of terms is higher than the normal three, this model has been successfully applied to other wide-angle-lens cameras without compromising object space reconstruction accuracy [29]. For the underwater calibrations reported herein, the 4th and 5th radial lens distortion parameters were not significant, so only three parameters were used. The displacement in x and y can be calculated with

$$\Delta x_k=\frac{\overline{x}}{r}\Delta r_k$$

(6)

$$\Delta y_k=\frac{\overline{y}}{r}\Delta r_k$$

(7)

Any lateral CA existing in a camera is predicted to be visible as differing radial displacements in the imagery for the different spectral bands. Any longitudinal CA that exists will be realised as different principal distances for the different colour bands.

In addition to radial lens distortion, there often is decentering distortion. As pointed out by [19] alignment errors between the optical axis and housing port can be absorbed as decentering lens distortion parameters. This distortion is asymmetric about the principal point and is normally much smaller in magnitude compared to the radial lens distortion. Decentering distortions can be modeled using two parameters p₁ and p₂ per spectral band:

$$\Delta x_{dk}=p_{1k}\left(r^2+2{\overline{x}}^2\right)+2p_{2k}\overline{xy}$$

(8)

$$\Delta y_{dk}=p_{2k}\left(r^2+2{\overline{y}}^2\right)+2p_{1k}\overline{xy}$$

(9)

Note that the i and j subscripts have been dropped for clarity reasons from the radial lens distortion function as well as all further distortion functions.

3.2. Least-Squares Adjustments and Constraints

The parametric (or Gauss–Markov) adjustment model was used for the self-calibration. All IOPs are modelled as network invariants. Datum definition was with inner constraints on object points. All unknowns (EOPs, IOPs and object points) were estimated simultaneously. Constraints added to the solution were implemented as weighted constraints according to the unified approach to least squares [30].

Three different least-squares solutions were implemented, which are introduced in

Table 1. Overview of the used adjustment types (left column), the abbreviation used in this paper (centre column) and a brief explanation (right column).

Independent	(IDP)	No constraints are added. Self-calibration of all bands independently.
Combined	(CMB)	Constraining that the locations of common object points to be the same.
Exterior Parameter Constraint	(EPC)	Constraining the camera locations and orientations (EOPs) of the different spectral bands images in addition to the CMB constraint

. The first adjustment type, referred to be the independent adjustment (IDP), performs a self-calibration of all colour bands separately. No constraints are added to this adjustment.

Each of the two additional adjustments included object space constraints. The combined adjustment (CMB) constrains that the locations of object points observed in each band are the same. In this adjustment three sets of IOPs (one per colour band) are estimated for each camera. Separate EOPs are estimated for each colour band image.

Finally, the EOPs of each spectral band for a given image are constrained to be equal. These constraints are added to the common object points constraint (EPC) adjustment, which is denoted as EPC. The EPC adjustment forces the red, green and blue EOPs to be equal. This constraint is formulated the way that the base vector (b_x, b_y, b_z) between the two bands is assumed to be equal to zero. For instance, the constraint between the red and green band positions can be formulated as

$$\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)+\left(\begin{array}{c}{\widehat{v}}_{bx}\\ {\widehat{v}}_{by}\\ {\widehat{v}}_{bz}\end{array}\right)=M_B\left(\begin{array}{c}b{x}_R^G\\ b{y}_R^G\\ b{z}_R^G\end{array}\right)$$

(10)

A similar equation is used to enforce the equivalent base vector components between the red and blue images. The relative angles between the pairs of images are also constrained:

$$0+{\widehat{v}}_{{\mathsf{\Delta }\mathsf{\omega }}_{BG}=arctan\left(\frac{-{\mathsf{\Delta }\mathrm{m}}_{32}}{{\mathsf{\Delta }\mathrm{m}}_{33}}\right)}$$

$$0+{\widehat{v}}_{\mathsf{\Delta }{\mathsf{\varphi }}_{BG}=arcsin\left({\mathsf{\Delta }\mathrm{m}}_{31}\right)}$$

(11)

$$0+{\widehat{v}}_{{\mathsf{\Delta }\mathsf{\kappa }}_{BG}=arctan\left(\frac{-{\mathsf{\Delta }\mathrm{m}}_{21}}{{\mathsf{\Delta }\mathrm{m}}_{11}}\right)}$$

where the Δm elements are extracted from the relative rotation matrix, shown here for the blue and green images.

$${\mathsf{\Delta }\mathrm{M}}_{\mathrm{BG}}={\mathrm{M}}_{\mathrm{G}}{\mathrm{M}}_B^T$$

(12)

The EOP constraints were enforced with high weights for each of the two groups of constraints: perspective centre position and relative angles. The standard deviations were 0.002 mm and 0.0002°, respectively.

Note that for the in-air calibration, strong fisheye lens distortions are expected [31] and as a result, a fisheye camera model [32] may be more appropriate to use for the in-air calibration. Nevertheless, strong distortions are unlikely for the underwater calibration with a planar port [18,23]. As the collinearity can be used to account for the high distortion by introducing additional polynomial radial lens distortion parameters [32], the collinearity models as introduced above have been used for all adjustments performed as part of this publication.

4. Datasets

Four GoPro Hero 5 Black cameras (1/2.3″ CMOS; 1.53 µm pixel spacing; 4000 × 3000 pixel captured image size; 2.68 mm nominal focal length) were used to investigate the CA and its effects. The cameras were installed on a frame that can be mounted on a remotely operated vehicle (ROV) (Figure 2). Accordingly, the cameras are labeled Front and Back as well as Left and Right. For instance, BL means the left back camera. Each camera is placed in a water-tight acrylic housing with planar port and mounted on the previously mentioned frame prior to performing the data capture.

For the underwater as well as in-air calibrations, the GoPro cameras were set to wide-angle mode and time-lapse photo acquisition mode. The image capture is synchronised using a remote control triggering to start the camera at the same time. Small time delays during the data capture are negligible for our tests. The cameras were not stopped or paused when moving from the underwater calibration field to the in-air calibration field, aiming to keep the relative positions of the cameras to each other constant. Unfortunately, the frame mount was not as rigid as required for constraining the relative orientation stability of the cameras [26].

The cameras stayed sealed in their housings during the image capture. The underwater and in-air images were captured in quick succession without altering the camera-housing setups in any way. Therefore, following the suggestion from [19], the cameras in their housings are seen as one “unit”.

The cameras were moved around each of the calibration frames whilst maintaining a constant standoff distance. The movement was performed through 120° about the target array for the underwater dataset and about 130° for the in-air dataset. The entire assembly was also rotated by 90° to provide roll diversity. In addition, every effort was made to fill the entire format with targets. These measures provided good geometry for the self-calibration adjustments. Many images of the calibration frames were taken for each calibration sequence. Some images suffered from motion blur and were excluded from any further analysis. An overview of the number of images used in the adjustments is provided in

Table 2. Least square adjustment results of the independent (IDP) adjustment for all in-air datasets showing for each dataset the #images, #object points observed, the degree of freedoms (DoF) and the Root Mean Square (RMS) values. Only one repeat of BL-A is shown. All other BL-A repeats achieved similar results.

In-Air	FL-A			BL-A
Bands	R	G	B	R	G	B
#images	10	10	10	10	10	10
#object pts	159	159	159	155	157	158
DoF	1484	1440	1430	1510	1478	1455
RMS (μm)	1.06	0.94	0.94	0.90	0.88	0.83
In-Air	FR-A			BR-A
Bands	R	G	B	R	G	B
#images	10	10	10	10	10	10
#object pts	160	159	158	158	159	158
DoF	1525	1618	1501	1395	1404	1451
RMS (μm)	0.77	0.80	0.77	0.79	0.79	0.84

and

Table 3. Least square adjustment results of the independent (IDP) adjustment for all underwater datasets showing for each dataset the #images, #object points observed, the Degree of Freedoms (DoF) and the RMS values. Only one repeat of FL-W is shown. All other FL-W repeats achieved similar results.

Water	FL-W			BL-W
Bands	R	G	B	R	G	B
#images	15	15	15	14	14	14
#object pts	139	139	139	147	148	144
DoF	1498	1482	1460	1354	1553	1323
RMS (μm)	1.65	1.63	1.73	1.27	1.31	1.34
Water	FR-W			BR-W
Bands	R	G	B	R	G	B
#images	14	14	14	14	14	14
#object pts	140	136	140	141	140	146
DoF	1631	1513	1551	1560	1555	1531
RMS (μm)	1.32	1.22	1.36	1.41	1.34	1.87

.

For the underwater images, the cameras mounted in the frame were submerged in a freshwater tank containing a target array. The calibration frame comprising 133 white targets (5 mm diameter) on a black background and is identically with the underwater calibration frame used by [26] (Figure 3, left). The distance between the camera stations and the frame was approximately 1.5 m.

The field of view of the camera is reduced considerably underwater due to the introduction of the planar port. As a result, it was not possible to use the same target field for the underwater and in-air calibrations. To ensure that control points covering the whole image, a larger in-air calibration field was utilised. Two poles and a scale bar placed in front of a wall ensured that the calibration field was not planar (Figure 3, right). The calibration field comprises approximately 155 retro-reflective, circular targets (5 mm diameter). The distance between the camera stations and the calibration field was approximately 2 m during the data capture.

For each camera (FL, FR, BL, BR) three repeats were captured in both environments (underwater (W) and in-air (A)); i.e., FL-A, FR-A, BL-A, BR-A, FL-W, FR-W, BL-W, BR-W. For the data processing, all three repeats of FL-W(1-3) and BL-A(1-3) were processed.

Each dataset contained at least 10 images. From each of the datasets, the red, green, and blue bands were extracted, and the image point observations were performed for each image independently applying centroid fitting using the Australis software (version 8.33). The observations of both datasets nearly covered the entire image plane.

Each dataset was processed using the three adjustment models: IDP, CMB and EPC. The adjustment results of the IDP method are presented in Table 2 and Table 3; the results are representative for all other adjustments. Overall, it can be summarised that the number of object points (ground control points) have been very high for all tests leading to high redundancies. The Root Mean Square (RMS) value is usually around or just below 1 µm for the in-air calibration which leads to the conclusion that the adjustment could successfully account for the strong fish-eye distortion of the camera. The RMS value for the underwater calibration is usually around or just below 1.5 µm and therefore slightly worse compared to the in-air calibration. Nevertheless, for the underwater calibration, the results show that the calibrations were successful, too.

Maximum correlations between the key parameter groups are summarised in

Table 4. Summary of the maximum correlation coefficients (in absolute values) from the 12 in-air and 12 underwater independent self-calibration adjustments.

	Maximum Correlation
Parameter Group	In Air	In Water
Principal point offset and perspective centre coordinates	0.86	0.80
Principal point offset and rotation angles	0.79	0.78
Principal distance and perspective centre coordinates	0.63	0.81
Radial lens distortion coefficients and perspective centre coordinates	0.09	0.28
Radial lens distortion coefficients and principal distance	0.73	0.34
Decentering lens distortion coefficients and rotation angles	0.40	0.77

. The results for the in-air and underwater calibrations are very similar and demonstrate the effectiveness of the network design measures employed for the experiments. High correlations (>0.9) were found between the radial lens distortion coefficients in both media. Such correlations are also reported for the wide-angle Ladybug5 camera [29], which exhibit similar high barrel distortion as the in-air calibration of the GoPros. However, these were found to have no detrimental impact on the independent accuracy assessment, but exclusion of distortion terms did degrade accuracy.

5. Evaluation

In this section, firstly the focus is to compare the IOPs of four cameras in-air and underwater. The analysis includes a comparison of the principal distance, principal point offset, radial and decentering distortion and aims to quantify the CA and the impact of the different medium on the IOPs. Secondly, the effectiveness of various constraints (IDP, CMB, EPC) of a representative camera was investigated covering the same IOP as mentioned previously. Then, the reliability of those IOP parameters of a single camera was investigated comparing the results of three repeats. The aim of this test was to investigate how stable the camera is. These tests were similar to [20], but explicitly evaluated the CA. Finally, an accuracy assessment of distance measurements and the impact of CA on those distance observations was investigated simulating the impact of CA on real world applications.

5.1. Test 1: Comparison of Underwater Calibration vs. In-Air Calibration

Firstly, the calibration parameters and the presence of CA were investigated using all four cameras (BL, BR, FL, FR). The CMB adjustment case was utilised. This Section contains firstly the analysis of the principal distance, followed by the analysis of the principal point offset, the radial lens distortion and the final decentering distortions.

5.1.1. Principal Distance

The results of the different principal distances are shown in Figure 4 and

Table 5. Principal distances and their standard deviations from the CMB adjustments.

Medium	Camera	Principal Distance
		Red (mm)	σ (µm)	Green (mm)	σ (µm)	Blue (mm)	σ (µm)
Underwater	BL	3.633	0.8	3.631	0.8	3.627	0.8
	BR	3.569	0.9	3.567	0.9	3.563	0.9
	FL	3.569	1.0	3.568	1.0	3.565	1.0
	FR	3.570	0.8	3.570	0.9	3.566	0.8
In-Air	BL	2.684	0.5	2.684	0.5	2.683	0.5
	BR	2.675	0.5	2.675	0.5	2.675	0.5
	FL	2.674	0.5	2.673	0.5	2.674	0.5
	FR	2.674	0.4	2.674	0.4	2.674	0.4

. The first column in Figure 4 shows the results of the four cameras (BL, BR, FL, FR) for the underwater environment. A clear trend is visible that the principal distance of the blue band is always the shortest principal distance, and that the principal distance of the red band is always the longest principal distance. The same observations have been made by [26] and agree with the model in Figure 1 (left) if longitudinal CA is present. The standard deviations of all principal distances are shown in Table 5. The statistical significance of the differences between the principal distances of the different bands was tested. The red–green and red–blue estimates’ differences were normalised by the standard deviation of the difference at 95%. Five of the eight principal distance pairs from the CMB adjustment were significant. It can be concluded that the longitudinal CA is present in the underwater images.

In contrast, the right column in Figure 4 shows the principal distances of the adjustments based on the in-air images taken from the four different cameras. The principal distances for the different bands for each camera are often identical. Statistical testing revealed that only one of the eight principal distance pairs were significantly different from the CMB adjustments. Therefore, the conclusion is that no longitudinal CA is present in the in-air datasets.

To summarise, the same cameras that show the effect of longitudinal CA in the underwater imagery showed no longitudinal CA in the in-air imagery. Noting that both (the in-air and underwater) datasets were captured with the presences of the plane port, hence, the change from air to water influences the behaviour of the light of rays causing CA in one medium but not the other. Consequently, the medium is the main contributing factor for the presence of longitudinal CA in the images.

5.1.2. Principal Point Offset

Then, the location of the principal points was investigated; the results of the calibrations are presented in Figure 5 and

Table 6. Principal point standard deviations from the CMB adjustments.

Medium	Camera	Principal Point
		Red		Green		Blue
		σxp	σyp	σxp	σyp	σxp	σyp
Underwater	BL	1.0	1.0	0.9	1.0	1.0	1.1
	BR	1.1	1.1	1.1	1.1	1.1	1.1
	FL	1.2	1.4	1.2	1.4	1.3	1.5
	FR	1.0	1.1	1.0	1.1	1.0	1.1
In-Air	BL	0.3	0.5	0.3	0.4	0.3	0.4
	BR	0.3	0.4	0.3	0.4	0.3	0.4
	FL	0.3	0.5	0.3	0.5	0.3	0.5
	FR	0.2	0.3	0.2	0.3	0.3	0.3

. Again, the left column of the Figure shows the results of the four cameras in the underwater environment. The colour of the dots indicates the band to which the observation belongs. The variation of the principal point location is approximately 3 µm in all cases. The standard deviations of all principal point offsets are shown in Table 6. Only three of the 16 principal point coordinates were statistically different at the 95% confidence level.

The right column of Figure 5 shows the results of the in-air calibrations. Compared to the underwater calibration, the locations of the principal points are more tightly clustered, within approximately 1 μm for all cases. None of the differences are statistically different. A trend indicating the largest offset based on the band is not visible. No final conclusion about the principal point offsets and any correlation to existing CA can be made.

5.1.3. Radial Lens Distortion

The analysis of the radial lens distortions can be used to investigate any lateral CA which is indicated by large differences in the displacements between the bands. The underwater calibration results (Figure 6, left column) show overall relatively small magnitudes of distortion but with significant differences between the bands. Red has always the smallest radial displacement while blue has the largest displacement. This agrees with the expectations shown in Figure 1 and with the observations made by [26]. The differences between the displacements of the colour bands indicates significant lateral CA. Indeed, this lateral CA is so strong that it is even possible to be seen in the images. Figure 7 show clearly shows that red has the smallest displacement and is oriented towards the centre of the image. In contrast, blue has the largest displacement and the direction is away from the image centre. In contrast, the right image in Figure 7 shows points located at the centre of the image, and no CA is visible for those points.

The in-air calibration results (Figure 6, right column) show very large radial lens distortions with magnitudes a thousand times bigger than in the underwater environment. The large displacements are due to the fisheye lens effect of the cameras in air, which disappears when placing the cameras underwater. Even when accounting for the different scales from in-air and underwater due to the different field of views (FoV) of the same camera in the different environments, the displacements in-air are still very large. The vertical line in Figure 6 (right column) indicates the influence of the different scales; the values show in italics next to the line indicate the displacement values of this distance. However, even though the displacements are large, significant differences between the different bands are not visible. It must be highlighted that the optics of the cameras were unchanged between the underwater and in-air image captures. Hence, it can be concluded that there is no lateral CA present in the in-air imagery.

To summarise, the same cameras which show the effect of lateral CA in the underwater imagery show no lateral CA in the in-air imagery. Hence, similar to the conclusion made for the longitudinal CA, the change from air to water influences the behaviour of the light of rays causing CA in one medium but not the other. Consequently, the medium is a main contributing factor for the presence of longitudinal CA in the images of the investigated cameras.

5.1.4. Decentering Distortion

Finally, Figure 8 presents the results of the decentering distortion parameters. The displacements are very small in magnitude for the underwater environment (left column) as well as for the in-air environment (right column). The differences between the different bands are not significant and no systematic trend can be observed with regards to which bands have the largest displacement.

5.1.5. Conclusions

The conclusion is that while significant lateral and longitudinal CA can be observed for the underwater datasets of different cameras, no CA was found in the in-air images of the same cameras. A separate analysis of the impact of the flat port housing vs. no housing vs. dome port housing is not included. For details about the effect of different housings (flat port and dome ports) in an underwater environment, we referred to [18,23]. As the housing was unchanged, the change from air to water influences is the main source for the changing of the behaviour of the light of the rays causing CA in one medium but not the other. Consequently, the medium is a main contributing factor for the presence of longitudinal CA in the images.

5.2. Test 2: Underwater: Effectiveness of the Various Constraints on the EOPs

The second test investigates the effectiveness of various adjustment constraints. As CA could only be observed in the underwater images, this Section only presents the results of one of the cameras (camera BL) for the underwater environment. However, any other cameras could have been used as the results of all cameras are very similar and show the same trends.

5.2.1. Principal Distance and Point Offset

Figure 9 shows the results for the principal distance (left column) and principal point location (right column). The first row presents the results of the IDP adjustment, followed by the results of the CMB adjustment in row two and the EPC adjustment in the third row. The IDP as well as the CMB adjustment show the trend that the principal distance of the blue band is the shortest, followed by green and red. While the explicit standard deviations of the principal distances and offsets are not presented, the precision of the IOPs from the IDP adjustments was an average of 46% lower (i.e., worse) than that of the CMB adjustments. Four of the eight principal distance pairs from the IDP adjustment were found to be statistically different at the 95% confidence level. Five of the eight principal distance pairs from the CMB adjustment were significant. As previously mentioned, this agrees with the findings in [26] as well as with Figure 1. The EPC adjustment, which forces the EOP parameters to be identical, breaking from the trend. Here, red has the shortest principal distance followed by blue and green. Three of the principal distance pairs from the EPC adjustment were significantly different. It can be concluded the longitudinal CA is present in the underwater images when applying the IDP, CMB and EPC adjustment constraints.

Focusing on the principal point locations, there is a systematic trend visible for this specific camera. The location of the principal point of the blue band is always furthest away from the origin of the image coordinate system, and that the green band’s is closest to the origin of the image coordinate system. Adding the constraints forces the principal points to be closer to each other. The differences in the principal point locations in the IDP adjustment are approximately 3 μm in the x direction and approximately 18 μm in the y direction. Four of the differences are significantly different. The location difference is reduced to approximately 2 μm in the x direction and approximately 2 μm in the y direction in the CMB adjustment. Only three of the 16 principal point coordinates were statistically different at the 95% confidence level. The differences of the principal point locations is even smaller (~0.1 μm) for the EPC adjustment. None from the EPC adjustment are significantly different.

5.2.2. Distortions

Figure 10 shows the results for the radial (left column) and the decentering lens distortion displacement (right column). For all adjustment types, the smallest radial distortion can be observed by the red band, followed by the green and blue bands. The difference in the displacement from red to green is approximately 13 µm and the displacement from green to blue is slightly larger with approximately 13 to 19 µm. The same trend is visible in all three adjustment results and indicates significant lateral CA. Overall, the different adjustment constraints do not influence the radial lens distortion profiles.

The magnitude of the decentering distortion is much smaller and always stays nearly under 5 µm and therefore, is marginal compared to the radial lens distortion. Again, the different constraints do not impact the results significantly.

5.2.3. Conclusions

The EPC constraint adjustment forces the principal distance and the principal point offset to be similar and indicate no longitudinal CA. In contrast, the IDP and CMB adjustments suggest the existence of longitudinal CA. The change in the adjustment constraints has no significant influences on the radial or decentering distortions, indicating the presence of lateral CA for all adjustments.

5.3. Test 3: Repeatability of Interorior Orientation Parameters (IOPs)

The aim of the third test is to investigate if the IOPs are repeatable. In other words, is the camera stable and can the CA influence be reproduced? As test one did not find any significant differences in the behaviours of the four tested cameras, only the camera FL for the underwater repeat sets and the camera BL for the in-air repeat were used for this test. Based on the results presented in test 2, the CMB adjustment results are presented in this Section only. As shown in test 2, the main difference between the CMB and EPC adjustment results are that the results of the principal distances and the principal point offset of the different bands are more homogenous. However, the distortion parameters for the CMB and EPC adjustment are comparable.

5.3.1. Principal Distance

Figure 11 presents the results of the repeats of the FL camera underwater (left column) and the BL camera in-air (right column). From the three repeats, two sets of parameter differences can be evaluated for each band for significance at the 95% confidence level. Four of the six pairs of principal distances were significantly different from the underwater calibrations, while only one was different for the in-air calibrations. While the differences between the bands seem to be very stable for each of the repeats of the underwater dataset, the actual principal distances vary between the repeats, indicating a less stable camera IOP. This does not agree with the results found in [20] with an older version of the camera (GoPro 3).

The results produced for the in-air dataset are very similar between the different repeats showing no significant differences and therefore indicating a very stable camera. CA is not visible in the in-air dataset.

5.3.2. Principal Point Offset

The comparison of the location of the principal point offsets for three repeats are presented in Figure 12. The left column presents the underwater repeats and the right column presents the in-air results.

The underwater repeats show again (similar to test 1 and 2) that the location of the principal point of the blue band is furthest from the image coordinate origin while the location of the principal point of the green band is again closest to the image coordinate origin. While the relative distances between the locations of the principal points are comparable and no more than 3 μm apart, the differences in the absolute location of the principal point are significant. Five sets of principal point coordinates were significantly different from the underwater datasets. This behaviour was not observed by [20], who used a previous version of a GoPro camera (series 3). The absolute locations of the principal point in-air appear to be more stable and are more tightly clustered, with differences of less than 1 μm. However, seven sets were significantly different from the in-air calibrations.

5.3.3. Radial Lens Distortion

Figure 13 summarises the results of the underwater (left column) and in-air (right column) adjustments of three repeats with regard to the radial lens distortion parameters. As previously discussed, the underwater images show a lateral CA expressed by the large differences in the radial displacements between the different bands. The displacements of the blue band at the edge of the image are nearly twice that of the magnitude of the red band. The displacement results between the different repeats is stable for the underwater calibration.

Then, the magnitudes of the displacements are much smaller for the underwater environment (the maximum displacement is smaller than 50 μm) compared to the displacement in-air (maximum displacement is just below 4700 μm). The vertical line in Figure 13 (right column) indicates the influence of the different scales; the italic values next to the line indicates the displacement values when accounting for the different scale due to the change in medium. Even when accounting for the different scales, the displacements in-air are still large (more than 110 μm). However, even though the displacements are large, significant differences between the different bands are not visible. Therefore, no lateral CA or even a trend for which band has the smallest or largest displacement is visible in the in-air datasets.

5.3.4. Decentering Distortion

Figure 14 presents the results of the decentering distortions for the underwater environment (left column) and in-air environment (right column). The same observations made for the radial distortion can be made for the decentering distortions. Regarding the underwater environment, this means that the same bands always show the smallest (blue) and largest (red) displacements and that the magnitude of the displacement is small and comparable (no significant difference). Regarding the in-air calibration, this means that there is no systematic trend of which band shows the largest displacements. However, compared to the radial displacements for the in-air calibration, the magnitude of the decentering displacement is very small.

5.3.5. Conclusions

The principal distances and the principal point offsets of the underwater repeats show significant differences. While the values are significantly different for each band between the repeats, the relative differences between the bands seems similar. The average RMS value for the image point positioning underwater is 1.6 µm for the CMB adjustment (and 1.7 µm for the EPC adjustment). In contrast, the principal distances and the principal point offsets in-air seem to be very stable. The average RMS value for the image point positioning in-air is 0.9 µm for the CMB adjustment (and 1.0 µm for the EPC adjustment). The radial and decentering lens distortion displacements are very similar for each repeat and appear stable both underwater and in-air.

As the same cameras are used in the same housings and as the camera settings are unchanged, the differences in the principal distance and the principal point offset in the underwater dataset cannot be caused by changes within the camera but the medium in which the images are taken. Therefore, it can be concluded that there is the requirement of in situ camera calibration for each underwater data capture and a recalibration of the camera for each new dataset captured mainly due to the medium in which the camera is operated (underwater).

Alternatively, it is suggested to use the EPC adjustment. While the radial and decentering lens distortion displacements are similar between the different adjustment types, the EPC adjustment constrains the principal distances and the principal point offsets to be stable. The same conclusion has been made by [26].

5.4. Test 4: Precision and Accuracy Assessment

5.4.1. Precision

Based on the conclusion of test 2, the CMP adjustment was used for investigating the precision of the EOP estimation in this Section. In this adjustment, each spectral band image has a separate set of EOPs and there are no conditions to enforce the fact that they were captured from the same position and in the same orientation. In contrast, the EPC adjustment adds adjustment constraints to force the EOPs of each band to be the same. As a by-product, the principal point coordinates for each spectral band are effectively the same. The common EOP element condition needs to be enforced with constraints, otherwise the EOPs from each band will differ due to random errors and CA. There are 6 constraints per pair of images (three for the relative position and three for the relative angles), six for R-G and six for R-B. Not enforcing these constraints means the camera positions can differ by several millimetres and the angles can differ by several arcminutes.

For each medium (water and air), all the EOP differences are provided for each dataset in

Table 7. EOP differences (min, max, RMS) of all cameras underwater (UW) and in-air when applying the CMB adjustment.

	Cameras	BL		BR		FL		FR
		Position	Angles	Position	Angles	Position	Angles	Position	Angles
		(mm)	(dd)	(mm)	(dd)	(mm)	(dd)	(mm)	(dd)
UW	Min	−4.1	−0.090	−4.4	−0.129	−4.1	−0.113	−4.3	−0.024
	Max	3.8	0.079	3.5	0.139	3.2	0.035	3.9	0.076
	RMS	1.8	0.031	1.9	0.061	1.7	0.030	1.7	0.024
Air	Min	−0.2	−0.019	−0.2	−0.016	−0.3	−0.029	−-0.2	−0.018
	Max	0.2	0.038	0.3	0.010	0.4	0.024	0.3	0.006
	RMS	0.1	0.015	0.1	0.005	0.1	0.010	0.1	0.008

. There is no CA effect in air, so the differences are overall very small and the RMS values are at the same level as the EOP precision. The EOP precision values are presented in

Table 8. Mean EOP precision of all cameras underwater and in-air when applying the CMB adjustment.

	Camera	BL	BR	FL	FR
Underwater	Xc (mm)	0.3	0.4	0.4	0.3
	Yc (mm)	0.4	0.4	0.5	0.4
	Zc (mm)	0.3	0.4	0.4	0.4
	omega (dd)	0.019	0.022	0.025	0.022
	phi (dd)	0.018	0.020	0.022	0.019
	kappa (dd)	0.010	0.012	0.014	0.014
Air	Xc (mm)	0.1	0.1	0.1	0.1
	Yc (mm)	0.1	0.1	0.1	0.1
	Zc (mm)	0.1	0.1	0.1	0.1
	omega (dd)	0.014	0.009	0.010	0.008
	phi (dd)	0.008	0.008	0.008	0.006
	kappa (dd)	0.011	0.006	0.007	0.005

. For instance, the position RMS for all air calibration is 0.1 mm, which is the same value for the precision results. In contrast, the position RMS for the underwater calibrations is between 1.7 and 1.9 mm compared to the precision which has a maximum value of 0.4 mm. The angular values mirror these findings. While this is not an exact comparison since the precision of the differences should be propagated, it is a good indicator for detecting any systematic and significant influences. The in-air RMS values are just due to random error. In contrast, the underwater calibration RMS values are 3x–6x higher than the EOP precision, so these values contain both random error and the systematic effects from the CA.

5.4.2. Accuracy

Then, the accuracy was investigated. For this test, the focus was purely on the underwater datasets. There are three reasons to exclude the in-air images in this test. Firstly, all previous tests have shown that the in-air datasets are not affected by CA whereas the main aim of this paper is to quantify and investigate the influence of CA. Then, the purpose of the system development is for underwater measurement and the in-air tests have been mainly used to better understand the impact of the IOP when changing the medium from in-air to underwater.

The aim of this test was to simulate a real-world underwater feature pickup of points as is done when surveying shipwrecks or infrastructure. Stereo-photogrammetry is often applied for such surveys, and the cameras are pre-calibrated. The stereo camera set up is ether achieved by mounting the two-camera system into a large housing, or by installing the cameras’ fixed frame. As the installation of our cameras on a frame was not solid enough, the alternative of removing an image pair from each underwater calibration was applied. The remaining images were used to estimate the calibration parameters.

The selected stereo image pair ensured good base separation, but a high convergence angle could not always be achieved due to the calibration set up and the limited space in the water tank. The extracted image pair was used in a separate bundle adjustment using the IO parameters determined with the larger subset of images for the accuracy assessment.

To overcome the datum defect, seven parameters had to be fixed. Two different methods were used to achieve this. Firstly, six EOPs from one image were fixed, plus one distance on the inner frame was included as an observation. The one distance in object space could be a scale bar placed in the field of view of the cameras in a real-world application. We refer to this adjustment as 6EOP in the following descriptions. Alternatively, when using a scale bar, it is common to fix the distance between the cameras. Therefore, the second method holds six EOPs from one image fixed plus one base vector component from a second image is fixed (similar to a dependent relative orientation). This simulates a stereo-camera, which is also often used in real-world applications. We refer to this adjustment as 7EOP in the following descriptions. For each method, five bundle adjustments were performed in total as formerly mentioned (for the IDP constraint, three adjustments: B only, R only, R only, then one adjustment using the CMB and one adjustment using the EPC constraint).

Check points were used to assess accuracy. Only targets placed on the rigid, central frame of the calibration field (Figure 2) were used as check points, which were solved as tie points in a second independent bundle adjustment referred to as accuracy assessment bundle adjustment. The tie point coordinates were compared with the reference coordinates to determine the differences, from which the mean, RMS, min and max differences were computed. The mean tie point precision from each case is summarised in

Table 9. Mean tie point precision (3D). All results are shown in mm.

Case	IDP	CMB	EPC
6EOP	5.6	3.6	3.5
7EOP	9.2	6.0	5.8

. The CMB adjustment precision is superior to that of the IDP cases by 35% and is the same as for the EPC cases. The 6EOP test precision is superior to the 7EOP precision by 40%.

The results for the 6EOP method are presented in

Table 10. The 6EOP accuracy assessment results. All results are shown in mm.

		IDP			CMB	EPC
	3D	B	G	R
BL	MEAN	1.1	0.8	0.6	1.3	0.6
	RMS	1.5	1.2	1.2	1.5	0.9
	MIN	2.7	2.0	1.7	2.3	1.5
	MAX	0.8	1.9	2.6	0.7	0.6
BR	MEAN	4.0	0.9	0.9	1.3	1.3
	RMS	4.9	1.5	1.4	1.5	1.5
	MIN	6.5	2.9	2.3	2.1	2.0
	MAX	2.9	1.0	0.8	2.2	2.1
FL1	MEAN	6.3	6.5	9.9	7.8	7.5
	RMS	6.9	7.0	10.6	8.3	8.0
	MIN	7.6	8.0	12.3	9.3	9.2
	MAX	6.3	6.7	7.4	7.0	6.4
FL2	MEAN	12.8	7.6	8.0	10.9	8.0
	RMS	13.0	7.7	8.2	11.0	8.1
	MIN	14.8	8.6	10.1	12.7	9.5
	MAX	12.1	7.1	9.2	9.8	7.4

. The RMS values were derived by comparing known coordinates with those determined by photogrammetry. The number of available check points varied between 4 and 10, but in most cases, 8–10 check points were available. The only case where this was not possible was for the camera FR. Therefore, instead of using data from the camera FW, the second repeat of the camera FL was used. The different repeats are labelled with FL1 and FL2.

The previous tests showed the largest radial displacement due to CA for the blue band followed by the green and red ones. The impact of the CA was also visible in this test as well. The RMS values for the blue band were the largest (up to 13 mm) followed by the red (up to 10.6 mm) and green (up to 7.7 mm) for the cameras BL, BR and FL2. Only FL1 shows a different trend with an inverted order. The CMB adjustment usually produces RMS values as large as the blue band (the largest RMS is 11 mm) again for the cameras BL, BR and FL2 but not for the camera FL1. Only introducing additional constraints (EPC adjustment) improves the RMS values and reduces the largest RMS value to 8.1 mm and the smallest value of less than 1mm. Overall, these RMS values are very high compared to the precision that can be achieved (c.f. Table 9).

The results for the 7EOP method (holding 6EOPs from one image fixed plus one base vector component) are presented in

Table 11. The 7EOP accuracy assessment results. All results are shown in mm.

		IDP			CMB	EPC
	3D	B	G	R
BL	MEAN	2.7	3.6	1.5	2.8	2.4
	RMS	3.1	4.0	1.9	3.1	2.7
	MIN	2.9	3.4	1.8	2.9	2.3
	MAX	4.0	5.7	3.8	4.0	3.4
BR	MEAN	16.9	7.9	7.3	10.6	10.0
	RMS	17.3	8.1	7.5	10.8	10.2
	MIN	19.5	9.6	8.7	11.2	10.6
	MAX	13.2	7.4	6.6	9.8	9.3
FL1	MEAN	3.7	4.4	8.9	5.1	3.7
	RMS	5.0	5.3	11.1	6.7	5.3
	MIN	2.2	4.7	10.1	4.4	2.9
	MAX	7.4	6.4	17.5	11.1	9.1
FL2	MEAN	10.2	4.0	2.7	7.8	6.2
	RMS	11.9	5.2	3.8	9.4	7.3
	MIN	10.7	4.9	4.5	8.7	5.8
	MAX	19.8	9.0	6.5	15.4	12.3

. Like in the 6EOP method, the largest RMS value is produced by a blue band (up to 17 mm) when using the IDP adjustment method. However, the blue band does not always have the largest value; also, the red band produces a very high RMS, e.g., for camera FL1, up to 11.1 mm. Again, the RMS values of the green band are the smallest (up to 8.1 mm). Overall, the RMS values have increased using the 7EOP method compared to the 6EOP method when using the IDP adjustment, which follows the trend of the tie point precision (Table 9). The RMS values are also increased for the CMB adjustment (largest RMS is 10.5mm) and for the EPC adjustment (largest RMS value is 10.2mm) when using the 7EOP method compared to the 6EOP method.

5.4.3. Conclusions

The analysis of the precision shows that air RMS values are just due to random errors. In contrast, the underwater calibration RMS values are 3x–6x higher than the EOP precision, so these values contain both random error and the systematic effects from the CA. For the accuracy assessment, two method were used. One simulating the use of a scale bar and one simulating a stereo-camera. The scale bar method using the EPC adjustment produces the smallest RMS values, which are, however, significant.

6. Conclusions

The aim of this paper was to investigate the chromatic aberration (CA) effects in low-cost camera systems (several GoPro cameras) operated in an underwater environment. This is an increasingly important field of research thanks to the availability of these cost-efficient camera solutions together with the opportunity to mount them on small ROVs. More precise projects for capturing underwater objects are being undertaken by many different researchers. The most popular fields of underwater image capture with such systems are within the area of coral reef monitoring as well as shipwreck mapping.

The contributions of this paper include the comparison of in-air and underwater camera calibration parameters of the same cameras with a specific focus on quantifying any CA. The comparison of four cameras used for underwater and in-air calibration showed that significant lateral and longitudinal CA can be observed for the underwater datasets. However, no CA was found using the same cameras calibrated in-air. The change from air to water influences the behaviour of the light of rays, causing CA in one medium (water) but not the other (air). Consequently, the medium is a main contributing factor for the presence of CA in the underwater images.

Furthermore, the effectiveness of the various constraints on the EOP is investigated in both in-air and underwater images captured by the same cameras. Three adjustment constraints have been investigated. The IDP method does not add any constraints; all bands are calibrated independently. The CMB constrains the locations of common object points captured by different bands so that they are the same. Finally, the EPC constrains the camera EOPs of the different bands to be the same in addition to the CMB constraint. The EPC constraint adjustment resulted in the principal distance and the principal point offset to be similar and indicated no longitudinal CA. In contrast, the IDP and CMB adjustments suggest the existence of longitudinal CA. There was a clear presence of lateral CA for all adjustments, but the change of the adjustment constraints has no significant influence on the radial or decentering distortions.

Moreover, the repeatability of the calculated camera calibration parameters in the underwater environment was investigated. The principal distances and the principal point offsets of the underwater repeats show significant differences. In contrast, the principal distances and the principal point offsets in-air seem to be very stable. The radial and decentering lens distortion displacements are very similar for each repeat and appear stable—underwater and in air. As the same cameras have been used and as the camera settings were unchanged, the difference in the principal distances and the principal point offsets in the underwater dataset cannot be caused by changes within the camera but the medium in which the images were taken. Therefore, it can be concluded that there is the requirement of in situ camera calibration for each underwater data capture and a recalibration of the camera for each new dataset captured mainly due to the medium in which the camera is operated (underwater). Alternatively, it is suggested to use the EPC adjustment. While the radial and decentering lens distortion displacements are similar between the different adjustment types, the EPC adjustment constrains the principal distances and the principal point offsets to be stable. The same conclusion has been made by [26].

Finally, a precision and accuracy assessment analysing the distance observation underwater was performed by simulating a real-world application using the images and quantifying the effect of CA on these distance observations. The analysis of the precision shows that the RMS values of the in-air adjustment are just due to random error in contrast to the underwater calibration RMS values being 3x–6x higher than the EOP precision. Hence, they contain not only random errors but also systematic effects from the CA. For the accuracy assessment, two methods were used: one simulating the use of a scale bar and one simulating a stereo-camera. The scale bar method using the EPC adjustment produces the smallest RMS values which are, however, significant.

Overall, it can be concluded that the common approach to correct the refraction effect using absorption by the physical calibration parameters will lead to significantly different camera calibration parameters of the same camera in-air and underwater. While effects such as CA are often not present in in-air application due to the cameras being mainly designed to be operated in air, the change of the medium from air to water can introduce significant CA which will not only impact the precision achievable but also the accuracy.

Future work will investigate the CA in the underwater environment. This paper has shown large differences for different repeats for the same camera underwater. There are several possible reasons. One of them is a possible distance dependency. Therefore, a further test will be conducted to quantify the effect of the distance to the calibration frame on the calibration parameters and CA, and if possible, to model this distance dependency. Furthermore, it is best practise to calibrate the cameras in a tank before using it for a possible job, e.g., in ocean water. The influence of the different water qualities on the camera calibration and the CA is another focus of future research.