Evaluating the Probability of Detection Capability of Permanently Installed Sensors Using a Structural Integrity Informed Approach

Abstract

There is a growing interest in using permanently installed sensors to monitor for defects in engineering components; the ability to collect real-time measurements is valuable when evaluating the structural integrity of the monitored component. However, a challenge in evaluating the detection capabilities of a permanently installed sensor arises from its fixed location and finite field-of-view, combined with the uncertainty in damage location. A probabilistic framework for evaluating the detection capabilities of a permanently installed sensor is thus proposed. By combining the spatial maps of sensor sensitivity obtained from model-assisted methods and probability of defect location obtained from structural mechanics, the expectation and confidence in the probability of detection (POD) can be estimated. The framework is demonstrated with four sensor-component combinations, and the results show the ability of the framework to characterise the detection capability of permanently installed sensors and quantify its performance with metrics such as the ${\mathrm{a}}_{90|95}$ value (the defect size where there is 95% confidence of obtaining at least 90% POD), which is valuable for structural integrity assessments as a metric for the largest defect that may be present and undetected. The framework is thus valuable for optimising and qualifying monitoring system designs in real-life engineering applications.

1 Introduction

In order to design and qualify Non-destructive Evaluation (NDE) procedures it is necessary to evaluate the expected detection capabilities. Ultimately, the value of undertaking NDE is determined by the confidence in detecting defects and the resulting structural integrity assurance that may be inferred. Over the last decades, significant research effort has been invested in developing evaluation methodologies for manual inspection procedures [1,2,3], in particular to evaluate the probability of detection (POD) and probability of false alarm (PFA) of candidate defects through receiver operating characteristic (ROC) analysis.

Recently, there is a growing interest in using permanently installed sensors to monitor defects in engineering components. A major advantage of using permanently installed sensors as opposed to inspections arises from the ability to collect continuous, real-time measurements, which can provide valuable in-service information about the monitored component.

There are fundamental differences between the measurement modalities of conventional manual NDE inspections and permanently installed sensors, and consequently fundamental differences between the evaluation of detection capabilities. These differences may be categorised as either temporal or spatial. The majority of existing literature is related to the temporal aspect, which will not be considered in detail in this paper. The temporal element relates to the ability to collect near-continuous measurements and therefore defect detection should be considered as a time-domain problem. Examples of this include the statistical dependence of the measurements being collected [4, 5], compensation for fluctuating environmental conditions [6, 7] and the effect of degradation on permanently installed sensors [8, 9]. In this paper measurements will be interpreted in isolation, relative to a defect free measurement.

This paper focuses on the spatial aspect of the detection capabilities of permanently installed sensors. While many manual NDE inspection techniques rely on the ability to manipulate the sensor position and pseudo-optimise sensor location to possible defects (e.g. manoeuvre sensor into a position of maximum signal amplitude), permanently installed sensors may be positioned in a sub-optimum location with respect to the sought defect. The detection capabilities will be determined by the location-specific sensitivity of the measurement to the defect at the given location, but the location of the emerging damage is unlikely to be known precisely or be predicted deterministically and so it is necessary to use a probabilistic approach. The anticipated location of damage may be predicted from past experience or surveys or predicted based on structural considerations such as distributions of stresses. Such a probabilistic approach is consistent with the aim of estimating a probability of detection.

The effect of defect location on the detection capability of a permanently installed sensor has been studied in previous literature [10], and is similarly studied as an optimisation problem for sensor network placement in structural health monitoring [11, 12]. A recent study has also been conducted to adapt the POD assessment procedure of NDE inspection techniques for permanently installed sensors using model-assisted methods [13]. However, to the knowledge of the authors, an approach that combines this information with structural integrity information on the probability of defect location has yet to be developed. This paper proposes a generally applicable framework which aims to produce, for a specific sensor-component combination, standardised visualisations and metrics that characterise the detection capability of a permanently installed sensor.

The proposed framework is composed of three main stages:

• (1) Evaluating the spatial distribution of probability of detection (POD map);

• (2) Evaluating the spatial probability of defect location (PDL map);

• (3) Combining the steps 1 and 2 to evaluate the overall anticipated detection capabilities.

The framework will be demonstrated on illustrative numerical studies of sensor and component combinations. In structural health monitoring, there is a well-recognised compromise between area coverage and sensitivity. Sensors may interrogate either a small inspection volume and therefore have high sensitivity to any changes within that volume, or interrogate a larger volume which inevitably leads to a reduction in sensitivity. The choice of design point within this compromise is clearly dependent on the confidence in defect location and in turn the required area coverage. The illustrative examples are chosen as they show how the framework may be used to quantify the detection capabilities of a sensor-component combination and how to address the well-recognised compromise between area coverage and sensitivity.

The paper will be structured as follows. Section 2 describes the example problems used to illustrate the proposed framework. Sects. 3 and 4 describes how spatial POD maps and PDL maps are obtained respectively. Section 5 describes the proposed methods of combining the two maps to evaluate the overall detection capabilities. Section 6 presents the results of the example problems when analysed with the proposed framework; the results are subsequently discussed in Sect. 7. Finally, Sect. 8 provides a summary of findings.

2 Description of Example Problems

In order to demonstrate the use of the evaluation framework, two structural problems are presented, together with two separate sample permanently installed sensors, giving a total of four sensor-component combinations. The two structural examples are a rectangular beam undergoing three-point and four-point fatigue bending; these provide examples with high and low confidence in the location of damage respectively. A bulk-wave ultrasonic sensor is suggested as an example with high sensitivity but low area coverage, and a potential-drop sensor is suggested as an example with lower sensitivity but higher area coverage.

The examples described here are hypothetical and are chosen to best illustrate the value of the proposed framework for quantifying the performance of a permanently installed sensor for different operating conditions. More realistic models of monitored components and sensors may be substituted. This section describes these examples in more detail to further elucidate the challenge, before the framework is described in the following section.

2.1 Description of Example Structural Problems

Two beams, each with dimensions shown in Fig. 1, are to be exposed to either three- or four-point fatigue bending with equal maximum stress amplitude and load ratio, $R=0$, meaning that the top surface is always in axial-compression while the bottom surface is always in axial tension. The surface axial stress distributions along the length of the beams are shown schematically in Fig. 2. The triangular stress amplitude distribution of the three-point bending example indicates that the maximum stress amplitude is experienced only at the centre of the beam, whereas the trapezoidal stress distribution of the four-point bending example indicates the maximum stress amplitude is nominally uniform between the two loading points. The two cases therefore represent scenarios where the area over which damage is expected to initiate is small and large respectively. Assuming that the defect will occur on the surface of the component experiencing tension, we can expect the damage to initiate close to the centre line of the component in the three-point bending case, whereas we can expect the damage to initiate anywhere in the uniformly stressed section between the two loading points in the four-point bending case. The distribution of probability of damage location will be quantitatively assessed in Sect. 4.

Fig. 1

Geometry of the monitored beam. All dimensions in mm

Fig. 2

Loading conditions and a schematic stress amplitude distribution of the bottom surface of the beam under a three-point fatigue bending, and b four-point fatigue bending. All dimensions in mm

2.2 Description of Exemplary Permanently Installed Sensors

Two simulated permanently installed sensors are considered: a bulk-wave ultrasonic sensor and a potential-drop sensor. The ultrasonic sensor is schematically illustrated in Fig. 3. A 45° ultrasonic shear-vertical wave is created by coupling a 25 mm diameter transducer to a 36° Perspex wedge which is fluid-coupled to the monitored component. The wedge is directed so the ultrasonic beam projects parallel to the $xz$-plane and is positioned so the centre of the ultrasonic beam is coincident with the centre of the bottom surface of the component. The excitation signal was a 5-cycle Hanning-windowed toneburst at a centre frequency of 1 MHz. The presence of a defect would reflect the signal back to the transducer, the amplitude of the reflected signal is dependent on the size and location of the defect and is used to indicate the presence of a defect. In finite element simulations, the signal excitation is modelled as phased out-of-plane point forces on the top surface of the block to simulate the angled ultrasonic beam from the wedge. The reflected signal is evaluated by summing the phased surface displacements at the same points where the signal is excited.

Fig. 3

Schematic of the permanently installed bulk-wave ultrasonic sensor evaluated for monitoring

The potential-drop sensor is schematically illustrated in Fig. 4. Current injection and sensing electrodes are placed at the two ends of the component. By monitoring the injected current and resulting voltage the transfer resistance can be calculated. The presence of a defect would deflect current flow in the component, resulting in an increase in resistance. The expected increase in resistance is again dependent on the size and location of the defect and is used as an indication of whether a defect is present.

Fig. 4

Schematic of the permanently installed potential-drop senor evaluated for monitoring.

The ultrasonic and potential-drop sensors are chosen to represent sensors of high sensitivity and low area coverage, and a sensor of low sensitivity and high area coverage respectively. One could imagine the ultrasonic sensor would be very sensitive to defects at the centre of the component where the ultrasound is interrogating, while having negligible sensitivity to defects near the sides and ends of the component. Whereas the potential-drop sensor would have a reasonably uniform sensitivity across the whole surface of the component. This will be demonstrated and quantified in the following section.

3 Spatial Probability of Detection (POD) Map of Permanently Installed Sensors

The methodology of obtaining the spatial POD map outlined in this section closely resembles that of model-assisted probability of detection (MAPOD) as implemented in the CIVA software [14, 15], and is also similar to the adaptation for permanently installed sensors proposed recently by Tschöke et al. [13]. An aim of MAPOD is to evaluate the anticipated probability of detection, including uncertainty, of a measurement procedure by considering various sources of variability. The use of model-assisted methods is necessary since it would be impractical to conduct the large number of destructive tests required for a probabilistic analysis, especially as each would require samples with PIMS installed on them.

There is a wide variety of parameters that may contribute to the uncertainties in the signal output of a permanently installed sensor, and this has been studied in a wide range of literature [16,17,18,19]. Some examples of these parameters include defect characteristics (location, orientation, morphology), sensor characteristics (size, frequency, position, noise) and sensor-component interaction (lift-off, coupling). A statistical distribution may be assigned to each of these parameters, and by sampling these parameters accordingly, the anticipated distribution of signal output of the sensor may be evaluated.

The distinction in the present study is that the influence of the defect location will be studied separately. The anticipated sensor response is evaluated for a defect occurring at each discretised spatial location to populate a POD map, which can then be combined with the spatial probability of defect location to refine the overall detection capabilities.

The process of producing a spatial POD map of a permanently installed sensor may be summarised according to the following process, which will be discussed in more detail in the remainder of this section:

• (1) Define a nominal measurement configuration to establish the model.

• (2) For each possible defect position, evaluate the response of the sensor to both defect-free and defective cases and populate a map of nominal sensitivity.

• (3) Identify parameters that influence the response of the sensor and estimate statistical distributions for those parameters. By appropriate sampling of the parameters evaluate the resulting uncertainty on the nominal sensor outputs.

• (4) For each possible defect position evaluate the POD with conventional receiver operating characteristic (ROC) analysis to populate a POD map.

Repeating the above process for a range of defect sizes would then allow the evaluation of the detection characteristics of a permanently installed sensor as a function of defect extent. The detailed process of producing the spatial POD map for a crack-like defect for the two permanently installed sensors considered will be demonstrated in the following subsections.

3.1 Defining a Nominal Measurement Configuration

There are multiple ways to evaluate the anticipated signal output of the permanently installed sensor as a function of the relative position between the sensor and the defect. This may be approximated with analytical or empirically-derived ad hoc approximations, an example being the use of distance-gain-size (DGS) scales to estimate the sensitivity of an ultrasonic sensor as a function of the distance between the sensor and the defect [20]. In the present study a finite element approach is used.

A “nominal configuration” will have to be defined and modelled. In this study, this is defined as a smooth, square crack-like notch having zero axial extent with sides of length, $a$, that is normal to the length of the beam. The location of the defect is defined as the midpoint of the bottom edge of the square. A schematic illustration of this is shown in Fig. 5. The nominal sensor configuration was described earlier in Sect. 2.2.

Fig. 5

Schematic of the cross-section of the beam and the candidate defect modelled in this study

For illustration purposes, a length of $a=6$ mm will be used in this section, and the effect of varying the length of the defect will be discussed later in Sect. 6.2.

3.2 Evaluating the Map of Nominal Sensitivity

In the present study, sensitivity is defined as the signal arising from the presence of a defect relative to the signal from a defect free case. This is not to be confused with the sensitivity–specificity definition found in some literature [10]. By running multiple simulations where the candidate defect is moved to different positions, the expected signal amplitude may be estimated, populating the sensitivity map. The simulations for the ultrasonic sensor were conducted using the finite element software ABAQUS® [21]; the simulations for the potential drop sensor were conducted using the finite element software COMSOL Multiphysics® [22].

The outputs, $X$, of the two sensors considered are defined as follows. For the ultrasonic sensor, this would be the maximum amplitude of the reflected signal obtained with the Hilbert transform and normalized to the amplitude of a corner-echo reflection; an example signal is illustrated in Fig. 6. For the potential-drop sensor, the resistance measured is normalized to the resistance measured when the component is defect-free; the signal output is thus the normalized resistance minus one.

Fig. 6

Plot of an example signal obtained from the Finite Element model of an ultrasonic sensor reflected from the modelled defect. The amplitude is normalised to the maximum amplitude from a corner reflection. The dotted line shows the Hilbert transform of the signal. The sensor output, $X,$ is the maximum amplitude of the signal

The resulting sensitivity map of the ultrasonic and potential-drop sensor are shown in Fig. 7. Again, as mentioned in Sect. 2.1, only defects initiating from the bottom of the beam are considered in this analysis. The choice of simulated defect positions are shown as red crosses in Fig. 7. These simulations points are chosen to provide a general trend of the sensor sensitivity with candidate defects at different locations. Clearly an increased number of points would provide a more accurate sensitivity map at the expense of greater computation time. This is especially significant for the fully 3D, time domain ultrasonic simulations, with simulation time for each point in the order of hours compared to seconds for the potential drop sensor. To reduce the number of simulations required, points outside the field of view of the sensor are assumed to have an output of zero amplitude.

Fig. 7

Maps showing a the nominal signal amplitude of the ultrasonic sensor, and b the nominal change in normalised resistance of the potential-drop sensor for a $6\times 6$ mm defect. The red crosses indicate points where FE simulations are conducted (Color figure online)

3.3 Evaluating the Uncertainty in Sensor Output

The influence of the defect location is the focus of this study and has been considered separately in Sect. 3.2. It is useful to separate the effect of systematic parametric uncertainty and spurious random noise. In the present examples a non-exhaustive list of parameters that may result in a systematic uncertainty in the sensor response may be: defect orientation, defect shape, defect roughness, uncertainty in sensor location and uncertainty in the quality of sensor-component coupling. Random noise may be evaluated as the variation in signals arising from identical measurement scenarios (i.e. repeat readings of the same defect). It is possible to map the spatial variation in uncertainty, such as through distance-dependent degradation in temperature compensation of guided waves [23], though in many situations, including the present examples, the uncertainty may be assumed to be spatially uniform.

For simplicity and clarity of focus, only the variation in signal output as a result of defect location and random noise will be incorporated. The effect of multiple parameters other than defect location and random noise would likely need to be studied in practice. The choice of parameters for each specific engineering application can be determined with parameter sensitivity analysis methods as reviewed in [24].

In the present study, the signal output of an ultrasonic sensor, ${X}_{UT}$, is assumed to follow the Rice probability distribution,

$${X}_{UT}\sim Rice\left({\overline{X} }_{UT}, {\sigma }_{UT}\right)$$

(1)

where ${\overline{X} }_{UT}$ is the expected sensor output from simulations and ${\sigma }_{UT}$ is the shape parameter of the distribution as a result of random noise, which is approximately $0.01$ as evaluated from the results in [25]. What is used here is an approximation of the typical capabilities of ultrasonic measurement systems, and advanced signal processing and multiparametric techniques can be used to reduce uncertainty in measurements. Nevertheless, this should be determined accordingly in practice with the actual monitoring system used under its operating conditions.

The measured change in normalized resistance of the potential-drop sensor, ${X}_{PD}$, is assumed to follow the normal distribution,

$${X}_{PD}\sim Normal\left({\overline{X} }_{PD}, {\sigma }_{PD}\right)$$

(2)

where ${\overline{X} }_{PD}$ is the expected sensor output from simulations and ${\sigma }_{PD}$ is the standard deviation, assumed to be $2.8\times {10}^{-4}$. This is approximated from the anticipated resistance being of order $10 \mu\Omega$ in finite element simulations and a standard deviation of approximately 2.8 $n\Omega$, which is the capability of a state-of-the-art potential-drop system [26].

3.4 Evaluating the Spatial Map of the Probability of Detection

With the distributions of sensor output for both defect-free and defective cases defined, receiver operating characteristic analysis can be used to evaluate the probability of detection for an acceptable probability of false alarm. The process is illustrated in Fig. 8. The probability of detection, POD, is defined as the probability of the signal output of a sensor, $X$, being greater than a chosen threshold when a defect is present. The corresponding probability of a false alarm, $PFA$, is defined as the probability of the signal being greater than the same chosen threshold when a defect is not present. In mathematical notations,

Fig. 8

Schematic illustration of the distributions of signal output from a defect-free state, $P\left(X|\mathrm{no defect}\right)$ and the defective case $P\left(X|\mathrm{defect}\right)$. With ROC analysis, a threshold is set for a given false alarm rate, PFA, and the subsequent probability of detection, POD, is evaluated

$$POD=P\left(X>\mathrm{threshold}|\mathrm{defect}\right)PFA=P\left(X>\mathrm{threshold}|\mathrm{no defect}\right)$$

(3)

The threshold is typically chosen to meet a predefined requirement of a maximum allowable PFA. A PFA of ${10}^{-6}$ is used in this study, which is significantly lower than what is typically used with NDE inspections. This is needed for permanently installed sensors since frequent measurements are made, resulting in more possibilities for a false alarm to occur. Note that for the ultrasonic sensor, the output contains a vector of $p$ data points, and a false alarm is defined as any of the output data points greater than the threshold. Hence, by DeMorgan’s theorem [27], PFA of the ultrasonic sensor is given by,

$$PFA=1-{\left(1-f\right)}^{p}$$

(4)

where $f$ is the probability of a single data point being above the threshold given there is no defect.

By repeating the process for defects at different locations, the spatial POD map for the sensors can be produced. The results of the two sensors for a $6\times 6$ mm square defect is shown in Fig. 9. As seen from the results, the ultrasonic sensor has near perfect detection capabilities with POD close to unity at the centre of the beam where the ultrasonic beam is directed. However, the sensitivity quickly diminishes with defects located away from the limited area of ultrasonic interrogation. On the other hand, the potential-drop sensor has a relatively even coverage over the entire beam except for the ends of the beam where the electrical current has yet to spread out sufficiently.

Fig. 9

Spatial POD Map for $6\times 6$ mm defects of a the ultrasonic sensor, and b the potential-drop sensor

4 Probability of Damage Location (PDL) Maps

To quantify the detection capabilities of a sensor-component combination, the spatial POD maps produced in Sect. 3 would have to be evaluated in the context of the spatial probability of defect location. The aim here is to evaluate the probability of detection given a defect of a defined severity is present. It is possible that more than one defect will be present, but the analysis provided here only considers the influence of a single defect of the determined severity.

In real-life applications, the potential defect location on a component is generally not known deterministically. The defect location will be determined by a combination of the externally applied demand on the component resulting in distributions of stress, temperature or environmental conditions, together with the intrinsic material properties and condition of the component [28]. In the case of fatigue damage, the formation of fatigue cracks depends greatly on the stress and location of microstructural imperfections (e.g. dislocations) [29]. Given this uncertainty in where the “weakest link” of the component is, it is necessary to evaluate the location of a damage using a probabilistic approach.

The overall aim is to produce what is defined here as the probability of damage location (PDL) map, which is the map of where a defect is most likely to initiate given there is a defect somewhere on the component. This can be mathematically expressed as,

$$PDL\left(i\right)=\frac{{P}_{f}\left(i\right)}{{\sum }_{k=1}^{n}{P}_{f}\left(k\right)}$$

(5)

where ${P}_{f}\left(i\right)$ is the probability of the defect being present at location $i$, and $n$ is the total number of discretized locations considered in this analysis.

Several methods can be used to construct the PDL map, or in other words to obtain a map of ${P}_{f}\left(i\right)$ for the monitored component. Some of these include the use of finite element models, experiments, surveying from previous experiences, or simply the identified inspection zones that are often readily available in practice [3]. In this study, the probability of damage location is assumed to depend on the stress distribution of the component only, and the weakest-link theory by Weibull [30] is used to evaluate the PDL map. The theory was originally developed to evaluate the effect of specimen size on measured material strength. The method has since been applied to evaluating a range of damage mechanisms such as brittle fracture [31] and provide a non-local stress approach to fatigue assessment [32, 33] with commercially-available software based on the theory [34].

The theoretical analysis of the example problem in the present study using the weakest-link theory is summarized in the appendix; only the results are shown here. Overall, ${P}_{f}\left(i\right)$ for fatigue damage is given by,

$${P}_{f}\left(i\right)=1-{\left(1-\Phi \left(\frac{\mathrm{ln}\left(N\right)-{C}_{1}+{C}_{2}\mathrm{ln}\left({S}^{^{\prime}}\left(i\right)\right)}{\sigma }\right)\right)}^{\frac{A\left(i\right)}{{A}_{ref}}}$$

(6)

where

• $N=$ number of loading cycles experienced by the component

• ${S}^{^{\prime}}\left(i\right)=$ stress amplitude with mean stress correction using the modified Goodman relationship [35] experienced by the component at location $i$

• ${C}_{1}$ and ${C}_{2}=$ material constants for the Basquin law [36]

• $\sigma =$ shape parameter of the lognormal distribution which describes the uncertainty in the Basquin law relationship between ${S}^{^{\prime}}\left(i\right)$ and $N$

• $\Phi \left(\bullet \right)=$ cumulative distribution function of the standard normal distribution

• $A\left(i\right)=$ surface area of discretised location $i$

• ${A}_{ref}=$ surface area of the specimen used to determine ${C}_{1}$ and ${C}_{2}$

The parameters used in analysing the sample problem in this paper are summarized in Table 1. Clearly the parameters selected here are rough estimates, but they provide an indication of the form of the results to expect from the analysis. Results from actual experimental data can be implemented when evaluating applications in real life.

Table 1 Material parameters used in the analysis

The map of stress amplitude for the two loading cases is produced using 3-D finite element simulations. The maximum stress amplitude for the two cases were set to be equal at $110$ MPa (see Fig. 10 for the stress distribution). This map of stress amplitude is then used to produce the corresponding PDL maps using Eq. (6) and values in Table 1. The results are shown in Fig. 11. As expected, the area over which a defect is likely to occur is much greater in the four-point bending case compared to the three-point bending case, meaning that the area which the monitoring system needs to cover is much larger.

Fig. 10

Stress amplitude maps (in MPa) for a the three-point fatigue bending, and b the four-point fatigue bending case

Fig. 11

PDL maps for a the three-point fatigue bending, and b the four-point fatigue bending case

Note that in theory, the PDL map would vary with the number of loading cycles as ${P}_{f}\left(i\right)$ is a function of the number of loading cycles. This variation is however found to be minimal and therefore insignificant to the analysis demonstrated here; from Eq. (5), one could imagine that ${P}_{f}\left(i\right)$ would steadily increase for all the elements as the number of loading cycles increase and so the normalized $PDL\left(i\right)$ remains reasonably constant.

5 Overall Probability of Detection—Combining the Spatial POD and PDL Map

With both the spatial POD maps for the two permanently installed sensors and PDL maps for the two loading conditions evaluated, they can now be combined to evaluate the detection capabilities of each of the four sensor-component combinations. The aim here is to quantify the expected POD and the confidence in its value for each sensor for a given PDL map.

5.1 Weighted Statistical Analysis

For clarity, the weighted histogram analysis will be described generically using Fig. 12 before being applied to the context of the present paper. Consider the case where there are $n$ possible mutually exclusive events ($n=10$ in the illustration in Fig. 12), each with an index, $i$, an associated event value, $x\left(i\right)$, and probability of occurrence, $P\left(i\right)$ (Fig. 12a). We wish to evaluate the expected event value and the associated confidence.

Fig. 12

Illustration of constructing a weighted histogram. An event space of $i=1, \dots , 10$ events is constructed. a Each event has a corresponding event value and probability of occurrence. b A naïve, unweighted histogram is constructed based on the event values; their corresponding probability of occurrence is also shown, represented by the shaded area. c The final weighted histogram weighted according to the corresponding probability of occurrence of each event

The list of possible event values can be visualised with a histogram (Fig. 12b). For an unweighted histogram, each event will contribute to one of $k$ bins according to the event value. The resulting height of each bin is simply the number of events in each bin category, ${n}_{k}$; the sum of heights of this histogram would therefore be $n$.

This histogram cannot be used to accurately evaluate the expected event value and its confidence as it does not include any information about the probability of each event occurring; it would implicitly assume there is an equal probability of the event occurring. To evaluate the expected event value, each contribution to the histogram should be weighted according to its corresponding probability of occurrence, producing what is known as a weighted histogram [39]. The height of each bin in the weighted histogram, ${w}_{k}$, can be calculated by summing the probability of occurrence associated with each individual contribution to the histogram,

$${w}_{k}=\sum_{i=1}^{{n}_{k}}{P}_{k}\left(i\right)$$

(7)

where ${P}_{k}\left(i\right)$ is the probability of event $i$ in bin $k$ (Fig. 12c). The sum of heights of this histogram is the sum of the probability of all events and therefore equals unity.

To evaluate the confidence intervals of the expected event value, a cumulative plot of the weighted results is also needed. This is produced by sorting the events in ascending order according to the event values and plotting the cumulative weights (i.e. probability of occurrence) as a function of the event value.

In the context of this paper, the events would be a defect occurring at each discretised location, $i$, on the monitored component; the event value for each location is the corresponding POD of the permanently installed sensor, $POD\left(i\right)$; the probability of occurrence is the corresponding PDL from structural integrity information, $PDL\left(i\right)$.

5.2 Evaluating the Overall Detection Capabilities

The analysis above is applied to the results from the four sensor-component combinations evaluated in Sects. 3 and 4; the case of using the ultrasonic sensor to monitor the three-point bending beam to detect $6\times 6$ mm defects is illustrated here. Results for all four sensor-component combinations and of different defect sizes will be discussed in Sect. 6.

Figure 13 shows (a) the unweighted histogram normalised to the total number of discretised locations, and (b) the weighted histogram. The weighted cumulative plot is also shown here as the orange line. Compared with the unweighted histogram, the effect of weighting the histogram with the probability of damage location is most apparent from the disappearance of the peak at $POD=0$; the $POD=0$ bin of the histogram was populated by contributions from outside of the region interrogated by ultrasound, but in the three-point bending example the PDL map provides the information that the defect is very unlikely to occur in these low-sensitivity regions and so should be given a low weighting accordingly.

Fig. 13

Relative frequency histogram of $PO{D}_{i}$ for the ultrasonic sensor—a unweighted, and b weighted by the probability of damage location, $PD{L}_{i}$

Using the weighted histogram and cumulative plot, two metrics of detection capabilities can be used to evaluate and quantify the performance of a sensor-component combination. These are also illustrated in Fig. 13. First is the expected probability of detection, denoted as $PO{D}_{exp}$, plotted as the dotted black line in Fig. 13b. This is defined as the probability-weighted average in probability theory, which is the sum of all the possible outcomes, which would be $POD\left(i\right)$, multiplied by the probability of occurrence of each respective outcome, which would be $PDL\left(i\right)$.

$$PO{D}_{exp}=\sum_{i=1}^{n}POD\left(i\right)\times PDL\left(i\right)$$

(8)

The second metric is the $POD$ with 95% statistical confidence, denoted as $PO{D}_{95}$. This is plotted as the dotted black line in Fig. 13b. In other words, this is the minimum probability of detection of the sensor for 95% of the cases when a defect is present. This is evaluated simply as the $POD$ which the cumulative plot reaches $0.05$ or $5\%$.

6 Results

Using the methods demonstrated in Sect. 5, the detection capabilities of each of the four sensor-component combinations can now be compared. The results for a single defect size ($6\times 6$ mm) will first be presented in Sect. 6.1, followed by the results for varying defect sizes in Sect. 6.2.

6.1 Detection Capabilities for a Single Defect Size

The graphical results for $6\times 6$ mm defects for all the four sensor-component combinations are shown in Fig. 14. A table of $PO{D}_{exp}$ and $PO{D}_{95}$ at $PFA={10}^{-6}$ is shown in Table 2.

Fig. 14

Weighted histograms of the $POD$ for $6\times 6$ mm defects of the ultrasonic sensor (a, c, e) and the potential-drop sensor (b, d, f) for the four-point (c, d) and three-point (e, f) bending case. The unweighted histograms (a, b) are also plotted for comparison

Table 2 $Po{D}_{exp}$ and $Po{D}_{95}$ for all four of the sensor-component combinations for $6\times 6$ mm defects

The histograms for the potential-drop sensor are shown in Fig. 14b, d and f. Between the naïve unweighted case and the four-point bending case the few low-sensitivity locations at the extreme ends of the component are weighted to zero and the corresponding low POD bins in the histogram are suppressed. The sensitivity over the remainder of the component is relatively uniform, resulting in the cluster of results at around $POD=85$-$95\%$; there is little consequence to increasing the confidence in damage location within an already uniform sensitivity. As the sensitivity is relatively uniform, there is little uncertainty in the POD and therefore the $PO{D}_{exp}$ and $PO{D}_{95}$ values are in reasonable agreement.

The histograms for the ultrasonic sensor are shown in Fig. 14a, c and e. Due to the bi-modal nature of the ultrasonic sensor histogram, there is a great deal of uncertainty in the POD; depending on the location of the damage the POD will either be near-zero or near-unity. As the region where a defect may be expected to occur gets successively smaller between the naïve, four-point and three-point cases, increasingly many low-sensitivity locations are weighted to zero and the corresponding $POD=0$ peak gets increasingly suppressed. Despite this, for both the three-point and four-point structural cases, there remains enough possibility that a defect will occur in a low-sensitivity region that the $PO{D}_{95}$ value is poor; even in the three-point bending case there is a 5% chance that the POD will be less than 32.1%.

Results in Table 2 indicate the importance of evaluating the expectation and confidence in POD. For the three-point bending case, based on the expected value the ultrasonic sensor appears to outperform the potential-drop sensor. Conversely, the $PO{D}_{95}$ values indicate that for the ultrasonic sensor there is a significant possibility that a defect may occur in a poor sensitivity region and therefore may go undetected, whereas this is unlikely for the potential-drop sensor due to the greater area coverage. The $PO{D}_{95}$ results significantly penalise sensor-component combinations where there is insufficient area coverage. This can be visualised by plotting the map of $\left[1-POD\left(i\right)\right]\times PDL\left(i\right)$ as shown in Fig. 15 for the case of monitoring the three-point bending case with the ultrasonic sensor; this map highlights areas of insufficient POD with significant PDL. A wider ultrasonic sensor would reduce the problem of insufficient coverage at the edges as indicated, though the problem would remain for the four-point fatigue bending case.

Fig. 15

Plot of $\left[1-POD\left(i\right)\right]\times PDL\left(i\right)$ for monitoring the three-point bending case using the ultrasonic sensor, highlighting areas of insufficient sensitivity with significant probability of defect location

It is also worth considering the impact of sensor mis-positioning. Optimal performance is achieved when the region where damage is likely to occur is well aligned with the field-of-view of the sensor. If the field-of-view of the sensor was not aligned to the region where damage is likely to occur, the performance would naturally suffer; the high POD elements would be poorly weighted instead of the low POD elements.

The analysis presented here helps to identify the performance limiting aspects of a permanently installed sensor. For the ultrasonic sensor, despite the very high sensitivity in the interrogated region, clearly the performance is limited by poor area coverage; increasing the sensitivity or reducing the measurement uncertainty (for example by improving the noise performance) is unlikely to cause significant improvement in performance. On the other hand, the potential-drop sensor has sufficient area coverage for both structural cases and increasing the area coverage still further will lead to negligible performance increase; in this case the performance is limited by the sensitivity and/or measurement uncertainty.

6.2 Detection Capabilities as a Function of Defect Size

The proposed methodology can be repeated to evaluate the detection capabilities of different sizes to produce a plot of POD against defect size. Square defects with side length $a=2$ to $8$ mm is evaluated, and the results are shown in Fig. 16. The expected POD and its 5% and 95% bounds are plotted. It is common to quantify the detection capabilities of a sensor as the defect size for which there is 95% confidence that a POD of at least 90% can be achieved by the sensor; this is often referred to as the ${a}_{90|95}$ defect size [2, 40]. The ${a}_{90|95}$ is recognised as being particularly valuable for structural integrity assessments as a metric for the largest defect that may be reasonably assumed to be present and go undetected.

Fig. 16

Plot of $POD$ against $a$ for the four sensor-component combinations with confidence bounds and the estimated ${a}_{90|95}$

To evaluate the ${a}_{90|95}$ defect size for each sensor-component configuration, the $PO{D}_{95}$ results are fitted to the logistic function [41],

$$PO{D}_{95}\left(a\right)={\left[1+\mathrm{exp}\left({K}_{1}+{K}_{2}a\right)\right]}^{-1}$$

(9)

where ${K}_{1}$ and ${K}_{2}$ are the fitting parameters of the logistic function. Subsequently, the estimated ${a}_{90|95}$ defect size is the defect size at which $PO{D}_{95}=90\%$. The resulting ${a}_{90|95}$ for each sensor-component configuration is shown in Table 3.

Table 3 ${a}_{90|95}$ for all four of the sensor-component combinations

As anticipated, due to the relatively uniform sensitivity of the potential-drop sensor there is little uncertainty in the POD and therefore little discrepancy between the $PO{D}_{exp}$ and $PO{D}_{95}$ values. The ${a}_{90|95}$ achieved for both the three-point bending and four-point bending cases is the same at $6.0$ mm. The potential-drop sensor is not limited by coverage and thus the increase in POD is in essence solely dependent on the defect size; the larger the defect, the more disruption in current flow is caused and the greater the change in resistance measurement.

Conversely, there is significant uncertainty in the POD of the ultrasonic sensor and therefore a significant discrepancy between the $PO{D}_{exp}$ and $PO{D}_{95}$ values. The $PO{D}_{95}$ values are poor as a result of the possibility that defects may occur in the regions with very poor sensitivity. For the four-point bending case the $PO{D}_{95}$ value never exceeds zero; if a defect occurs in a location with negligible sensitivity then it will not be detected regardless of its size. This conclusion is subtly different for the three-point bending case, as indicated by Fig. 15 the locations with inadequate coverage are situated to the sides of the component. Consequently, when a defect increases in size it ‘grows’ into an area of high sensitivity and is therefore detected. Again, a wider sensor would overcome the problem of insufficient coverage for the three-point bending case.

7 Discussion

Quantitative assessment is necessary to optimise the selection and design of permanently installed sensors. It is proposed that the analysis described in this paper may be repeated for a range of different monitoring system design parameters in order to maximise the ${a}_{90|95}$ value. As an example, the ultrasonic transducer diameter or frequency could be altered in order to achieve sufficient area coverage, without compromising too far on sensitivity.

In the examples used in this paper, finite element structural analysis and weakest link theory is used to provide a probability of damage location specific to each discretised spatial location. It is worth emphasising that on many occasions such involved analysis is not required. As an example, a simplification of simply dividing the component into uniform domains of ‘likely’ and ‘unlikely’ areas for damage location would have little consequence on the conclusions in many cases; in the three-point bending case the ‘likely’ areas would correspond to an element wide strip down the centre of the component, while for the four-point bending case the ‘likely’ area would be between the two central supports. The difference in the results of $PO{D}_{exp}$ and $PO{D}_{95}$ for the simple case studied here would have been within $5\%$ compared to the more comprehensive study using FE simulations and the weakest link theory.

This paper focuses on the spatial aspect of POD and uses model-assisted methods to study the systematic effect of uncertain defect location on sensor sensitivity. The use of a model-assisted approach was necessary since conducting a large number of destructive tests for a probabilistic analysis would be impractical. This is especially true in this case as permanently installed sensors would need to be installed in each of the destructively tested samples.

Systematic effects other than defect location, such as loading and environmental conditions, can also have an effect on the POD or PDL map. In practice, a sensitivity analysis would be required to identify and study the effects of key parameters affecting the detection capabilities of the sensor and the structural integrity of the component [42, 43]. Given the modular nature of the approach proposed, it can easily be adopted to incorporate such analyses as needed for specific industries and use cases in real life.

This paper does not exploit the ability of a permanently installed sensor to collect frequent, real-time data, which may significantly improve its detection capabilities; the analysis presented here is only based on isolated measurements. As mentioned previously, it is useful to divide sources of uncertainty into systematic parametric uncertainty and incoherent random effects. For the purpose of simple illustration, only the effect of random noise is considered in the present study. While repeated measurements have the effect of averaging out and suppressing the influence of random incoherent noise, the detection capabilities may not improve ad infinitum due to environmental influence [44] and parametric uncertainty [4]. In real-life applications, this would have to be further investigated for each individual case in order to accurately quantify the detection capabilities.

8 Conclusions

A generally applicable, probabilistic framework for evaluating the detection capabilities of permanently installed sensor is proposed in the present study. The spatial aspect of the detection capabilities of a permanently installed sensor is addressed by combining a map of sensor sensitivity obtained using model-assisted methods with a map of probability of defect location obtained from structural integrity information.

The spatial map of probability of detection may be evaluated using a model assisted approach. There are many methods for estimating the probability of defect location; these range from simple segmentation into ‘likely’ and ‘unlikely’ areas of defects arising, to more quantitative structural mechanics-based assessments. In this paper a finite element-based weakest-link method was used.

Illustrative examples have been used to demonstrate the value of the evaluation framework; in particular in addressing the well-recognised area-coverage sensitivity compromise. Histograms of POD are suggested to be particularly valuable in evaluating the performance of the monitoring system, and when coupled with the spatial maps of POD and PDL can help identify the performance limiting design aspects.

Key performance parameters such as $PO{D}_{exp}$, $PO{D}_{95}$ and ${a}_{90|95}$ have been suggested and evaluated. The ${a}_{90|95}$ is recognised as being particularly valuable for structural integrity assessments as a metric for the largest defect that may be reasonable assumed to be present and go undetected. This paper demonstrates that the ${a}_{90|95}$ is particularly sensitive to inadequate area coverage. Quantifying performance is necessary to be able to optimise monitoring system design, and tailor it to the specific needs of the structural integrity assessment.

Appendix: The Weakest-Link Theory for Fatigue Damage

The weakest-link theory by Weibull [30] was originally developed to evaluate the effect of specimen size on measured material strength. The theory was subsequently applied to evaluate a range of damage mechanisms, including brittle fracture [31] and provide a non-local stress approach to fatigue assessment [32, 33]. Commercially-available software based on the theory have also been developed for fatigue analysis [34].

The underlying concept of the weakest-link theory is that the larger the component, the more potential there is for the component to contain flaws or aberrant material where damage are likely to initiate.

Consider a reference specimen of standardized dimension with surface area ${A}_{ref}$. The probability of survival (no life-limiting damage initiating) of the reference specimen is ${P}_{s, ref}$. By dividing the surface of the reference specimen into elements of equal surface area, $A\left(i\right)$, ${P}_{s,ref}$ can be expressed as the probability of all the individual elements on the reference specimen surviving (no life-limiting damage initiating on any of the elements), ${P}_{s}\left(i\right)$. In mathematical terms,

$${P}_{s,ref}={P}_{s}\left(1\right)\times {P}_{s}\left(2\right)\times \dots \times {P}_{s}\left({m}_{i}\right)=\prod_{i=1}^{{m}_{i}}{P}_{s,i}={\left({P}_{s,i}\right)}^{\frac{{A}_{ref}}{A\left(i\right)}}, {m}_{i}=\frac{{A}_{ref}}{A\left(i\right)}$$

(10)

Rearranging gives,

$${P}_{s}\left(i\right)={\left({P}_{s,ref}\right)}^{\frac{1}{{m}_{i}}}$$

(11)

The probability of failure (a life-limiting damage initiating) of element $i$, ${P}_{f,i}$, can therefore be defined as,

$${P}_{f}\left(i\right)=1-{P}_{s}\left(i\right)=1-{\left({P}_{s,ref}\right)}^{\frac{1}{{m}_{i}}}=1-{\left(1-{P}_{f,ref}\right)}^{\frac{1}{{m}_{i}}}$$

(12)

The significance of Eq. (12) is that it can be used to evaluate the probability of failure of an arbitrary surface using the results of a reference specimen. For instance, results from laboratory tests using standardized specimens can be applied to any component with stress field obtained from finite element results, and $A\left(i\right)$ becomes the surface area of an element. This comes with the additional benefit of evaluating how likely is a life-limiting damage going to initiate for each of the element on the component.

For fatigue damage, the probability of failure is mainly dependent on the equivalent stress amplitude corrected for mean stress effects, ${S}_{a}^{^{\prime}}$, and the number of loading cycles, $N$. By Basquin law [36], the expected number of cycles to failure, $\overline{{N }_{f}}$, is,

$$\overline{{N }_{f}}=\mathrm{exp}\left[{C}_{1}-{C}_{2}\mathrm{ln}\left({S}_{a}^{^{\prime}}\right)\right]$$

(13)

where ${C}_{1}$ and ${C}_{2}$ are material constants evaluated from the elastic modulus, the reduction in area in a tensile test, and the endurance limit of the material [35]. There is inevitably uncertainty in the actual cycles to failure, ${N}_{f}$, of a component. Assuming that ${N}_{f}$ is lognormally distributed, we can express Eq. (13) as,

$${N}_{f}=\mathrm{exp}\left[{C}_{1}-{C}_{2}\mathrm{ln}\left({S}_{a}^{^{\prime}}\right)+z\sigma \right]$$

(14)

where $z\sim Normal\left(\mathrm{0,1}\right)$ and $\sigma$ is the shape parameter of the lognormal distribution. There is a range of literature discussing the most appropriate distribution type for fatigue damage, such as lognormal [45], Weibull [46], or generalised extreme value distribution [47]. A lognormal distribution is used here as this is common practice and most applicable in engineering design with readily available data for common materials in standards [35, 45].

To evaluate the probability of failure for any given combination of ${S}_{a}^{^{\prime}}$ and number of loading cycles, $N$, Eq. (14) can be rearranged to give,

$$z=\frac{1}{\sigma }\left[\mathrm{ln}\left(N\right)-{C}_{1}+{C}_{2}\mathrm{ln}\left({S}_{a}^{^{\prime}}\right)\right]$$

(15)

Here, $z$ can be seen as a generalised parameter which follows the standard normal distribution that define the contour lines of equal probability of failure in the ${S}_{a}^{^{\prime}}$-$N$ space as illustrated in Fig. 

Fig. 17

Plot of ${S}_{a}^{^{\prime}}$ against $N$ with selected contour lines of probability of failure, ${P}_{f,ref}$

17. The cumulative probability of failure of a standard specimen, ${P}_{f,ref}$, is thus simply the cumulative distribution function of the standard normal distribution, denoted as $\Phi \left(z\right)$. Substituting this into Eq. (15) gives,

$${P}_{f}\left(i\right)=1-{\left(1-\Phi \left(z\right)\right)}^{\frac{1}{{m}_{i}}}$$

(16)

With this, the probability of failure of an arbitrary surface, $i$, can be evaluated with Eq. (16) if the following is known:

• Area of the surface, $A\left(i\right)$

• Material constants, ${C}_{1}$ and ${C}_{2}$ for Eq. (13) evaluated with a specimen of standardized geometry with surface area, ${A}_{ref}$

• Equivalent loading amplitude experienced by the surface, ${S}_{a}^{^{\prime}}$

• Number of loading cycles experienced by the surface, $N$

Several assumptions are made when applying the weakest-link theory to fatigue damage in this analysis. Firstly, as stated above, it is assumed that fatigue damage manifests at the surface of a component. This is a valid assumption if no significant voids are present within the material. This assumption is especially valid in the case of bending fatigue as the maximum stress amplitude experienced by the beam is at its bottom surface. In the case of other damage mechanisms, the volume of an element instead of surface area is usually considered [31]. It is also assumed that life-limiting damage initiating from the component does not interact. This is a valid assumption for high-cycle fatigue where the number of life-limiting damage is small and thus sparsely distributed [33].