2.1.

Collagen Labeling

Type I collagen solution (5026-50ML, TeloCol, Advanced BioMatrix) was used for the study and labeled with AF488 (Alexa Fluor 488 TFP ester, A37570, Invitrogen) as described previously.^11,14 Briefly, collagen monomers were diluted in 10 mM HCl to $1\mathrm{mg}/\mathrm{mL}$ and then mixed 1:1 with 0.2 M sodium carbonate–bicarbonate buffer (24095, Polysciences) containing 1 M sodium chloride (S671-3, Fisher Scientific). AF488 was dissolved in dimethyl sulfoxide (D12345, Life Technology) to a concentration of $0.5\mathrm{mg}/\mathrm{mL}$ and added to collagen solution in excess amount to achieve at least two fluorophores on average on each collagen monomer. The reaction mixtures were stirred for 3 h at room temperature. Labeled monomers were dialyzed in dialysis cassettes (Slide-A-Lyzer, 3.5K MWCO, 3 mL) for 3 days in 10 mM HCl. Collagen concentration was measured using DC Protein Assay (Bio-Rad). AF488 concentration was measured using Beer–Lambert law. Degree of labeling was measured as moles of fluorophore per mole of collagen.

2.2.

Optical System

The sample was illuminated with a blue LED light source [ThorLabs M470-L3-C5, 470 nm, 650 mW (minimum) LED]. The filter set used was a Nikon Fluorescence Cube (96311 B-2E/C). The excitation filter and dichroic reflection on this cube provide a pass-band excitation filters of blue light (465 to 495 nm wavelengths) as shown in Fig. 1(a) (data provided by the manufacturer). Wavelengths that pass through the filter and are reflected by the dichroic mirror are focused onto the sample with a $60\times$ objective lens (Nikon’s CFI Apochromat TIRF Series, numerical aperture: 1.45) and fluorescent emission is detected through the same lens. Fluorescence from the sample is transmitted through the dichroic and emission filters with a pass-band for green light (515 to 555 nm wavelengths). Most of the emission spectrum is passed through this filter as shown in Fig. 1(c). From Fig. 1(b), we note that the transmission of any scattered or reflected excitation light at 470 nm is reduced by nearly 10 orders of magnitude. The microscope was a Nikon inverted microscope (ECLIPSE TE2000-E) that used two cameras: CoolSNAP EZ CCD Camera ($1392\times 1040$ imaging array with $6.45\text{-}\mu \mathrm{m}$ square pixels). The images obtained by the cameras were taken using the Nikon NIS-Elements software.

Fig. 1

Transmission and reflection spectra of the Nikon 96311 B-2E/C filter cube that is obtained from the manufacturer. (a) Light from the source is transmitted through an emission filter and reflected by the dichroic mirror. (b) Light emitted by the sample is transmitted through an excitation filter and dichroic mirror. (c) Emission through the excitation spectrum that is obtained from the manufacturer of the AF488 fluorescent dye.

2.3.

Image Simulation

The CoolSNAP EZ CCD camera has a resolution of $6.45\mu \mathrm{m}$ per pixel, with a $60\times$ objective the resolution results in 110 nm per pixel. A collagen monomer at 300 nm long only subtends a few pixels at best. However, due to diffraction, a fluorophore, which can be considered a point object, is imaged as the PSF, of diameter:

The first step of the simulation was to determine the number of electrons the sensor produces from signal, background, scattered light, and noise. The radiance spectrum of the blue light source (470 nm) was multiplied by the transmission through the excitation filter and reflection through the dichroic filter [Fig. 1(a)]. Then, the above spectrum was converted to photon spectral irradiance rate using the photon energy and the solid angle of the objective lens, in which the solid angle $\Omega$ was calculated by

We then simulated the number of electrons in the sensor from the scattered light and background noise. In our simulation, we estimated a background level of light caused by excitation light that passed through the dichroic and emission filters to reach the detector. The background was considered as a number of photons per pixel and could include other sources such as out-of-focus fluorophores and autofluorescence. To model leakage of the excited light, a similar process was used except that instead of calculating fluorescence from absorption and quantum yield, we multiplied the photon irradiance by the scattering section of the AF488 dye, estimated as $1.1{\mathrm{e}}^{-13}{\mathrm{m}}^{-2}$. The rest of the process is the same, but now the wavelength is still that of the source, so the transmission through the emission filters is lower. Background light can also be added. The noise is determined by the exposure time multiplied by the readout RMS and dark current values provided by the manufacturer plus electrons from scattering. It is important to note that these signal and noise values do not account for the autofluorescence of the sample, scattered light that leaks in the camera, and residues of fluorescent material.

To generate a synthetic image, a random pixel position was chosen to emulate one of the fluorophores with the value of the signal in electron counts, in a resolution 10 times smaller than the original dimensions. To simulate a double-labeled monomer, a second fluorophore was located at a random distance from the other, at a maximum of 300 nm down to 150 nm, with a random orientation between [90 deg, $-90\mathrm{deg}$] in intervals of 15 deg. We chose to limit the minimum separation of fluorophores to 150 nm. The pixel size dictated by our camera and objective is 110 nm, so we do not expect reliable measurements of double-labeled monomers below this limit. Nevertheless, we will image about half of the double-labeled monomers, providing a useful sample of monomer orientation. With random locations for the second fluorophore, about half of the double-labeled monomers would be included within these dimensions, enabling a statistical analysis of orientation in a sample of collagen monomers. The image was convolved with the PSF that represents the optical transfer function. The image was then resized to its correct dimension. Finally, a Poisson distribution was applied to the total number of electrons, resulting in the final image (Fig. 2).

Fig. 2

Simulation for collagen monomers: first generation of fluorescence point sources and defining a PSF determined by our optical setup. Then, the point sources are seen by the diffraction-limited system after the aperture. The resulting image representing the accumulation of photons in the camera is corrupted with Poisson noise. Result in a dataset labeled by different orientations including singularity monomers. Colormap indicates electron counts.

2.4.

Bank of Gabor Filters for Determination of Orientation in Monomers

Gabor filters are linear filters that are well-suited for quantification of given spatial frequency components at specific locations and orientations in a two-dimensional signal.^15,16 Figure 3 shows the steps of the algorithm we devised to estimate monomer orientation with Gabor filters. A simulated input image, with one double-tagged monomer is enhanced by the nonlinear “block-matching and 3D filtering” denoising method.¹⁷ Samples of the filters in the bank are chosen [Fig. 3(a)] and show all feature maps resulting from convolution of the enhanced image and all sample filters [Fig. 3(b)]. Gabor feature is the result of the product of the image with the filter, being the value different for each of the filters in the bank. The maximum value of the Gabor feature is plotted for each orientation [Fig. 3(c)]; the maximum degree is shown in perpendicular orientation. The final orientation estimation is superimposed upon the input image [Fig. 3(d)]. A Gabor filter cannot detect a monomer that is singularly labeled, as the pattern is expected to be circularly symmetric, and thus all the filters will have similar results except for variations caused by the Poisson distribution and noise. For this reason, an initial decision must be made based on the area to discern if the monomer is one or two labeled. To determine pixels that are part of the monomer area, a value for each pixel of 175 counts is selected as threshold. If the value is over this threshold it can be considered as part of the signal and not noise or background, being able to calculate the area of a monomer. For the modify Gabor filter, if the area has less than seven pixels above threshold it will be classified as a single monomer.

Fig. 3

Steps for detection of orientation with a bank of Gabor filters. (a) Montage of real part of a bank of Gabor filters with wavelength of 5. (b) Magnitude of Gabor output for the results of applying all the sample filters to a monomer image. We will also call this the Gabor feature, which is a function of spatial position in both axes and the orientation of the specific filter in the bank. (c) Polar plot of maximum Gabor feature for each orientation, finding the degree for the orientation. (d) Input monomer image and superimposed position and detected orientation. Detected orientation is 15 deg, followed by 2.1891 being the maximum value of the corresponding Gabor feature for the overlap of the image with the filter. Colormap indicates electron counts.

2.5.

CNN for Detection of Monomer Orientation

In another algorithm to measure the angle of orientation of the monomers, a machine learning approach was taken. A CNN was trained and tested on 26,000 samples of simulated data, divided into 70% for training and 30% for testing. A validation dataset conforms of 6500 samples is used to measure the results in Sec. 3.1.2. The task was trained as a classification problem, with monomers labeled as being within one of 12 classes, each with an angular range of 15 deg. There is also a 13th class that denoted single-tagged monomers. These are monomers that only contain one fluorescent tag and therefore an orientation cannot be determined. We used two convolutional layers with a categorical cross-entropy loss function for the classification task with a calculation of 52,909 parameters. The model was trained in three epochs and the number of samples was 32 for the batch size. Architecture of CNN is described in Fig. 4. Training was made in Python3.6 and Keras2.3.1 in Intel Core i7-8750H CPU @ 2.20 GHz 2.21 GHz machine. Total time in training was 6.87 s. Our use of CNN is distinct from traditional tasks this type of architecture is used to solve. Unexpected orientation of objects can typically fool object detection models, whereas our architecture is trained to measure this very orientation.

Fig. 4

Architecture of CNN having as components two convolutional layers.

3.1.

Detection of Monomer Orientation in Simulated Data

3.1.1.

Gabor results

The Gabor filter set has a performance of 55.03% accuracy and when it failed, the misclassification was usually next to the closest orientation leading to an error of 15 deg. If a monomer has a single fluorophore, the filter will try to detect an orientation anyway. The circular symmetry will be broken by pixel-to-pixel variations caused by the Poisson statistics or noise and one of the filters will incorrectly have the best match. To correct this deficiency, an initial decision based on area was made to distinguish between a double and single labeled monomer. For validations 6500 samples were used. The results of classification are presented in Fig. 5(a). Most misclassifications are close to the diagonal, so we can say that Gabor filter has an error of $\sim 15\mathrm{deg}$.

Fig. 5

Classification results of synthetic data. Show accuracy and misclassification of the two approaches. (a) Confusion matrix of the Gabor filter set. (b) Confusion matrix of the CNN.

3.2.

Experiments with Fluorescent Collagen Monomers

Next, we applied the two algorithms to data collected from images of labeled monomers, which were absorbed on glass in the laboratory. The samples were from a $314\text{-}\mu \mathrm{g}/\mathrm{mL}$ collagen solution with an average labeling of 2.5 tags per collagen monomer. Images were taken with an exposure time of 5 s. For single molecule visualization, samples were diluted 100 to 1000 times to prevent light from saturating the camera and to reduce the fluorescence and scattering from monomers that were out of focus. Monomers in the experimental image were clearly visible so the images were tested to determine orientation (Fig. 6).

Fig. 6

(a) An experimental image using fluorescence microscopy. (b) Five numbered regions of interest show fluorescence detection results for isolated collagen monomers. Region numbers correspond to the numbered locations in (a). Examples show the output of the classification of orientation of Gabor filter and CNN model in degrees.

3.3.

Detection of Monomer Orientation in Experimental Data

3.3.1.

Detection in isolated monomers

Assuming that all fluorophores were attached to collagen, collagen monomers were detected by the level of signal. In any given image, it was possible to detect several isolated monomers. As shown in Fig. 5, the Gabor filter is more likely to misclassify than the CNN model. One problem is that it is not possible to distinguish between single and double tagged monomers using just the Gabor filter so that decision was based on the area of the image. The most significant discrepancy between the two models can be observed in Fig. 7. When the algorithms differ, there is uncertainty as to which is more correct because, unlike the simulations, the experimental data lack ground truth. Epifluorescence measurement followed by sample preparation and electron microscope measurements would be a challenging task, probably requiring a special instrument with both imaging modalities.

Fig. 7

Confusion matrix for the two models. The results for the detection of 154 isolated monomers for Gabor filter and the CNN predictions.

3.3.2.

Sequential detection

In biomedical applications, collagen monomers are usually studied in physiological conditions and in association with collagen fibrils. To address the dynamic interaction of collagen monomers with fibril surfaces, different monomers were analyzed in successive images collected at different times to test the reproducibility of the algorithms under different realizations of Poisson electron statistics and noise. Eight pictures were captured separated in time by 10 s. There is a slight variation from one capture to the next, as the illumination and the noise vary. The results of the algorithms for every isolated monomer through the eight times were compared. Also, a visual classification for each monomer was made, with the purpose of comparing the similarity of the models to human eye observation. Observation was done by one of the authors. Monomers that are classified constantly in the same position through time are more likely to have a specific orientation. An important consideration is distinguishing between singularity and multiply labeled classification, as it is not possible to determine orientation to singularity labeled. The variation of orientation for double labeled classification is determined through the sequential frames calculating the standard deviation of degree. Singularity labeled classification is not accounted for this purpose because of no directionality. Detections from different monomers through time are shown in Fig. 8. The average standard deviation for double label monomers with detected degree of orientations was 37.79 deg for the Gabor filter, 38.77 deg for the CNN model, and 29.9 deg for the eye observation.

Fig. 8

Table of comparison for every timestamp for monomers in the same sample. Images were taken every 10 s. Orientation is classified between [−90 deg, 90 deg] for double labeled monomers and is shown as a continued function. Classification of singularity labeled monomers (SM) is shown as separated points, for the reason of no directionality. Results from the Gabor filter, the CNN model, and the eye observations are shown in blue, red, and green colors, respectively.

In addition to noise and Poisson fluctuations in electron count, there may be other reasons for differences between successive images. The most likely is photobleaching as the sample is excited several times. Measuring the difference of the sequential images pixel by pixel over the eight times sample shows that the average signal decreases by only 2.18 counts with a standard deviation of 13.5 counts. Because the signal is so similar in every frame substantial photobleaching is unlikely. Imaged monomers were all adhered (completely or partially) to glass. It is possible that some of those monomers were only partially adhered (e.g., only one end of the monomer was adhered to the glass, whereas the other end was free). This would introduce limited variability in the monomer orientation. Therefore, the variability of monomers orientation over time shown in Fig. 8, can be due to detection method and/or partial adherence of monomers to glass.