Introduction

The International Atomic Energy Agency (IAEA) supports its Member States to increase the utilization of their nuclear research reactors, and the IAEA Technical Cooperation Programme (TCP) plays a major role in transferring nuclear knowledge and technology to its Member States.

In addition to scientific research and training, research reactors are often initiated for providing commercial services and products, such as radionuclides for medical and industrial applications. An overview of opportunities for utilization of research reactors is available [1] from which it can be derived, as well as from the IAEA Research Reactor Data Base [2], that the neutrons from miniature, small and medium-size reactors are in many cases used for neutron activation analysis (NAA). Over the years, the IAEA has stimulated the orientation of NAA laboratories worldwide to relevant fields of application. Whereas the Member States may have identified markets for NAA laboratories, demonstration of valid analytical data and organizational quality of the work process are preconditions for consolidating and expanding the stakeholder community. For these reasons, laboratories and/or stakeholders may prefer that the facility’s management system is accredited for compliance with the International Standard ISO/IEC17025 [3], and several NAA laboratories already accomplished this.

One of the requirements in the process towards accreditation is that the laboratory provides objective evidence of the validity of its measurement results by, amongst others, participation in proficiency testing (PT) schemes by interlaboratory comparison. Participation in an interlaboratory comparison round may reveal that some results are not satisfactory. Laboratories then face the problem of finding the source of such non-conformity and applying, if relevant, effective corrective actions. Obviously, PT providers cannot deliver a laboratory and technique-specific after-care on potential sources of error and approaches to mitigate them.

The IAEA therefore has implemented since 2010 a new mechanism for supporting NAA laboratories by a three-tier approach to assist the laboratories in assessing their proficiency in analysing material of various composition and effectiveness of implemented quality assurance and quality control approaches by:

  1. 1.

    Facilitating participation in successive rounds from an (ISO/IEC17043, [4]) accredited PT provider, ensuring rapid issue of the evaluation report by the PT provider. This was accomplished in 2010, 2011, 2012, 2013, 2015 and 2017. Starting in 2018, the rounds are managed by the IAEA Nuclear Science and Instrumentation Laboratory (NSIL), which has decades of experience in organizing interlaboratory comparison rounds for proficiency testing of nuclear and nuclear-related analytical techniques.

  2. 2.

    Critical technical and analytical analysis of the results by an IAEA International Expert for indications on potential sources of error and initial feedback.

  3. 3.

    Feedback workshops of participants and IAEA International Experts shortly after availability of PT provider’s and IAEA Expert’s reports. These workshops included detailed discussions for identifying unanticipated sources of errors (both technical and managerial), approaches for elimination thereof and discussions on relevant methods for quality assurance and quality control. The aim of these workshops was to design with participants a path towards sustainable performance in analytical best practice. The workshops were held in 2011, 2012, 2013, 2015 and 2017.

The number of participating NAA laboratories increased from five, representing five Member States in 2010, to 41 from 29 Member States in 2018.

Method of implementation

The IAEA facilitated participation in the WEPALFootnote 1’s international soil and plant exchange programmes [5] for the rounds 2010–2017. Soil is considered to be an ‘easy’ material for NAA, whereas analysis of plant matrices may be more difficult given the much lower induced activities and risk of contamination. The number of elements to be reported is restricted by WEPAL’s reporting format, where not all elements are present. In the years 2010, 2012 and 2015 two such rounds were facilitated.

In 2018, the IAEA NSIL in Seibersdorf [6], with support of the IAEA TCP, provided and distributed two samples in an interlaboratory comparison round for proficiency testing: a marine sediment and an animal tissue. These samples mimic the analytical challenges for laboratories previously posed by the above-mentioned soil and plant material, respectively. Both materials had established degree of homogeneity and well characterized mass fractions of the major, minor and trace elements.

Participants submitted their results also to NSIL for further data evaluation. In this 2018 round, each result had to be accompanied by an estimate of its uncertainty expressed as one standard deviation. No restriction on the number of the reported elements was imposed, and therefore many uncertified elements were also reported.

Participants

Laboratories with highly experienced NAA practitioners as well as laboratories with newcomers in the technique participated in each regional (Africa, Europe, Asia and the Pacific, North America and Latin America) groups. These laboratories use different types of research reactors (varying from subcritical facilities, miniature reactors to medium-size reactors), differently configured gamma-ray spectrometers and different methods of calibration and data processing.

Data evaluation

WEPAL provided the IAEA an EXCEL file with all results of the IAEA facilitated NAA laboratories including WEPAL’s own routine evaluation based on z-scores [7].

For the 2018 interlaboratory comparison exercise, NSIL calculated z-scores, u-scores and density distribution functions for each element with an assigned value or with a consensus value on basis of participants’ results [8].

Typically, NAA laboratories reported mass fractions up to 25 elements in each sample of the soil-type materials, and up to 20 elements in each sample of the biological type. Sometimes the rescaled sum of z -scores, RSZ, is used as a singular indicator if more than one measurand is involved in a proficiency test:

\({\text{RSZ}} = \frac{{\mathop \sum \nolimits_{i = 1}^{{\text{N}}} z_{i} }}{\sqrt N }\) with N = Number of z scores and \(z_{{\text{i}}} = \frac{{x_{{\text{i}}} - x_{{{\text{PT}}}} }}{{\sigma_{{{\text{PT}}}} }}\) with xi the test result for the given measurand, xpt is the associated assigned value, and σpt is the standard deviation for proficiency assessment of the specific measurand [7].

The drawback of this approach is that z-scores of opposite sign cancel out, thus masking extreme |z| values. The IAEA evaluated the performance of each laboratory based on the fraction (as percentage) of the submitted data reported for which the absolute z-value, |z| < 3. This is in agreement with clause 9.9.4 of the ISO13528:2015 [8] in which it is stated that ‘In proficiency testing schemes that involve a large number of measurands, a count or proportion of the numbers of action and warning signs can be used to evaluate performance’. The performance of some NAA laboratories in previous interlaboratory comparison exercises indicated that under routine, i.e. day-to-day experimental conditions, at least 90 % of all reported data would meet this criterion, i.e. max. 2–3 outliers per sample. As such, the 90 % fraction was used as a target indicator for excellent performance of the NAA laboratories.

Satisfactory performance was assigned to laboratories with 70 %–90 % of their data with |z| < 3. It should be noted that the z-scores do not account for the laboratory’s own uncertainty of measurement. Many laboratories prefer therefore the zeta-score instead, which would often result in a smaller number of outliers, especially since the combined uncertainty of measurement may be equal to or larger than the standard deviation of the consensus value.

The laboratories with less than 70 % of their data with |z| < 3 were categorized as ‘in development’.

This performance indicator (fraction, as percentage of submitted data with |z| < 3) allowed for monitoring the development of the improvement of a laboratory in successive proficiency testing rounds. All performance indicators are for sake of comparison only and are not based on international conventions.

Feedback workshops

The feedback workshops were held as quickly as possible (with the exception for the first one) after completion of the last round in the respective periods (see Table 1).

Table 1 Feedback workshops held

Participating laboratories were encouraged to select their representative as the person(s) that actually carried out the analyses. This recommendation was well followed-up, which facilitated evaluation, stimulated discussions and resulted in individual, bottom-up action plans for improvement.

All participants presented, using a proposed IAEA template, details of their analytical procedure applied in the interlaboratory comparison testing. These details included, e.g. the sample masses, dry mass assessment, calibration procedure, corrections for neutron flux gradients, internal quality control applied and status of quality assurance implementation. In addition, participants provided their own view on their performance and, if relevant and possible, their hypothesis on sources of error.

The results were further discussed within the broad platform of participants, moderated by the IAEA International Expert and IAEA Technical Officer for these projects. One of the observations was that, especially in the rounds organized by WEPAL, mass fractions of elements such as Al and Cr measured by NAA are sometimes significantly higher than the median of all results in such interlaboratory comparison exercises. In a few cases, this has led to |z| ≥ 3. It is assumed that the corresponding consensus values are sometimes slightly underestimated, as the majority of the other participants use techniques requiring digestion of the test portion and may not reach complete dissolution of some elements [9, 10]. As such, the NAA results might be more realistic estimates of the true values of the total mass fractions of these elements than the consensus values. However, the difficulties with the complete dissolution of Cr also resulted in a relatively large standard deviation of the consensus value that compensated the bias of the NAA results with the consensus value in the z score. No corrections were therefore applied in the IAEA metrics.

Participants discussed several best practices in the different types of calibration in NAA, such as the selection of calibrators, neutron flux monitors and irradiation and counting geometry. Approaches were exchanged for managing nuclear reaction interferences by epithermal and fast reactor neutrons—such as for the measurement of the elements Al and Mg upon the use of the relative method of calibration in NAA, gamma ray spectral interferences and related peak fitting, like in the measurement of Sr via 85Sr. The related practical recommendations have been incorporated in the IAEA e-learning course on neutron activation analysis [11, 12].

The test portions used in NAA vary from tens of milligrams to about 200–300 mg. The participants questioned it if some deviating results, especially for trace elements at low mass fractions, might be caused by insufficient degree of homogeneity of the materials.

Several laboratories, either at newcomer facilities or with inexperienced staff had difficulties with trouble shooting and implementing corrective actions. To this end, the feedback workshops were complemented with lectures on relevant aspects of the analytical procedures, such as on details of calibration, spectral and other interferences, methodologies for internal quality control, method validation, use of control charts and sample preparation.

The lessons learned during these feedback workshops were used for drafting action plans for improvements to be implemented; in addition, they were again distributed as a reminder to the participants at each new round.

Lessons learned

The following main sources of error were identified in the feedback workshops:

  • Insufficient study of the associated documentation from the PT provider on the sample handling (such as estimation of the moisture fraction, and minimum sample mass to be analysed) and on the required dimensional units in the reporting.

  • Shortage of valid calibrators/control materials and use of expired calibrators, often due to a lack of financial resources. Such constraints forced some laboratories to use substantially smaller amounts than prescribed of, e.g. reference materials.

  • Absence of independent internal quality (validity) control such as by simultaneous processing of reference materials and blanks, and no own strict acceptance criteria if such control materials are processed.

  • Insufficient checking of results upon reporting resulting in transcription errors, reporting in different units than required, and/or exchange of samples and results. This was often ascribed to a too tight planning of the analyses. Many laboratories reported their results very close to the deadline—although an ample timeframe of about 3 months was given for completion of an analysis.

  • Differences in counting geometries of calibrator and standards.

  • No corrections for neutron flux gradients.

Almost all participants, upon awareness of deficiencies in their analytical procedure and laboratory organization, initiated implementation of corrective actions (see, e.g., [13, 1415]). This effected in improvements in the results in the next rounds. It is assumed that the analysis done in the proficiency testing rounds is representative for the quality of other ‘routine’ analyses performed by these laboratories.

Participants were reminded during the workshops that Member State laboratories can submit requests to the IAEA for procurement of calibrators and (certified) reference materials.

Outcome

An example of improvement in participant laboratory performance is given in Fig. 1. The improvement by Laboratory 1 is based on their own evaluation and corrective actions; the improvements in Laboratories 2 and 3 is based on the lessons learned during the feedback workshop in 2012. The effectiveness of their corrective actions is demonstrated by the acceptable performance in the rounds in following years.

Fig. 1
figure 1

Fraction (as percentage) of submitted data with |z| < 3 for selected NAA laboratories with increasing improvement of performance. ‘Bio analysis’ refers to the analysis of plant samples in 2010–2017, and animal tissue in 2018

After reaching a very high level of performance in 2012/2013, some decline in performance for soil samples by laboratory 1 can be noticed, although the performance is still in the ‘excellent’ range. The NAA data analysis and interpretation is in some facilities no fully automated and requires several human interactions and transposing actions. Some errors indicate indeed mistakes like interchange of data but it can also not be excluded that less experienced staff did the most recent analyses.

Laboratory 4 joined the project in 2015. Inexperienced staff did the analyses in the first round. The effect of the 2015 workshop is clearly visible in the improvement in performance; by 2018 they almost reached their target performance for the soil analysis (90 % of results with, |z| < 3). However, still a few improvements are needed for analysing material with low trace element values.

The current performance levels of laboratories 1–4 is nearly on par with that of several other NAA laboratories that reached a very high number of satisfactory results already at the start of this project (laboratories 5–7 in Fig. 2).

Fig. 2
figure 2

Fraction (as percentage) of submitted data with |z| < 3 for selected NAA laboratories with consolidated performance since 2011. ‘Bio analysis’ refers to the analysis of plant samples in 2010–2017, and animal tissue in 2018

The trends in the performance of all NAA laboratories participating in this project is depicted in Fig. 3 for both matrices: the relatively easy soil-type samples and the more challenging biological samples with low trace element levels.

Fig. 3
figure 3

Number of NAA laboratories participating in the successive IAEA facilitated interlaboratory comparison rounds from 2010 to 2018 of soil-type material and biological-type material, categorized to their level of performance based on percentage of results with |z| < 3 (see the section on data evaluation)

NAA is a mature technique, with a straightforward measurement equation [16], but in practice expertise in the technique and knowledge of all effects in quantification, which leads to avoidance of potential sources of error, comes from hands-on experience. Different to other analytical techniques, NAA is rarely a part of the analytical chemistry curriculum at technical schools or in academia. The IAEA therefore supports its Member States in this area via expert missions and fellowship training. However, the relatively frequent turnover of experienced staff in some laboratories hampers the training and transfer of knowledge to new staff, which may have consequences for analytical performance, especially if no quality management system has been implemented. These issues likely underpin laboratories categorized in Figs. 37 as ‘In development’. Such laboratories often have more difficulties with the low trace element level samples—and thus low induced activities, with lower count rates and resulting poor counting statistics. Further, experience is lacking on corrections for interferences, blanks, measurement geometries and coincidence summing effects, to name a few. In addition, absence of quality management also sometimes contributes to gross errors indicated in the section ‘Lessons learned’.

The differences in performance of NAA laboratories in the various IAEA regions Latin America and the Caribbean, Europe, and Asia and the Pacific can be derived from Figs. 4, 5, 6. There are, unfortunately, insufficient results from NAA laboratories in the regions Africa and North America for a meaningful comparison.

Fig. 4
figure 4

Number of NAA Laboratories in Latin America and the Caribbean and their performance according the IAEA metric in successive intercomparison rounds for soil type samples and bio-type samples

Fig. 5
figure 5

Number of NAA Laboratories in Europe and their performance according the IAEA metric in successive intercomparison rounds for soil type samples and bio-type samples

Fig. 6
figure 6

Number of NAA Laboratories in Asia and the Pacific and their performance according the IAEA metric in successive intercomparison rounds for soil type samples and bio-type samples

The comparably better performance by the NAA laboratories in Latin America and the Caribbean may to some extent be considered as the impact of many joint IAEA/ARCALFootnote 2 projects in this region with emphasis on implementation of quality assurance and quality control, and on the relatively less frequent staff turnover in those NAA laboratories resulting in consolidation and transfer of expertise.

The largest interlaboratory comparison among NAA laboratories was held in the year 2019 with the support of the TCP regional project ‘Enhancing Utilization and Safety of the Research Reactors’ [17]. Fifteen European laboratories operating activation analysis techniques registered for this proficiency test, making it the largest regional contribution out of the 46 NAA laboratories from 32 Member States worldwide. The results of a few first time participants were not yet levelled to those of the others in this region, so it is expected that the lessons learned from continuing participation will further improve the overall regional performance in Europe.

An increasing number of laboratories in the Asia and the Pacific region joined the project since 2015, and some of them had a disappointing performance at their first round (Fig. 6). For example, no laboratory qualified for the category ‘Excellent’ in analysing biological-type samples in 2018. The main reasons identified have already been mentioned in the above: relatively frequent staff turnover and insufficient implementation of quality assurance and quality control procedures.

In some cases, and in all regions, laboratories decided to have newly arrived staff or students perform the analyses, some of them with very little experience. These laboratories reported benefitting from participating under such conditions, as it gave them insight on critical areas, where improved quality assurance procedures and/or quality control mechanisms need to be implemented, and on the need to invest further in training. To support training, the IAEA E-learning course on neutron activation analysis [11, 12] includes ample guidance not only on the fundamentals but also on the practice and quality assurance/quality control of the technique, and is therefore widely used and acknowledged.

Continuation of participation will lead to an improvement in performance of the laboratories currently categorized as ‘Satisfactory’ and ‘In development’, as observed in other laboratories shown in Fig. 1.

Many laboratories favour a comparison of results on basis of zeta-scores, as this accounts for the uncertainty of measurement. The reporting of measurement uncertainty in the 2018 round allowed for a comparison of the performance classification on basis of z-score and zeta score as shown in Fig. 7. Using the zeta score ( and, similarly as for the z-score, the criterion |zeta| < 3) resulted in a larger number of laboratories within the best performance category ‘Excellent’, while reducing the number of those in the weakest category.

Fig. 7
figure 7

Number of NAA laboratories participating in the 2018 round, categorized to their level of performance on basis of z-score and zeta score. ‘Bio’ refers to the analysis of plant and animal tissue. One laboratory only analysed the soil sample, and not the bio sample

Conclusions

This IAEA initiative to facilitate laboratories’ participation in proficiency testing schemes complemented by a new approach of feedback workshops, expert missions and procurement of indispensable calibrators and reference materials, resulted in a measurable increase in the analytical performance of most participating laboratories. Several laboratories demonstrated consolidation of their already satisfactory performance over time.

A new metric has been introduced for assessing the performance of the participants, based on the fraction of all data reported with a performance |z| score < 3. Such a metric has not been published in reports of other interlaboratory comparison exercises; therefore, it is not possible to derive any conclusions from comparison to other analytical multi-element techniques.

The results from the interlaboratory comparison rounds confirmed that many laboratories operating NAA have reached an operational level at which they periodically report satisfactory to excellent results for the last 5 years. A few laboratories, representing ~10 % of all participants in each round, did not show much improvement from their first participation and continue their efforts for better performance taking advantage of the lessons learned during the feedback workshops.

Several NAA laboratories, especially in developing countries, have insufficient resources of their own to cover the fees of interlaboratory comparison testing schemes. Therefore, it has been advised to consider bi- or multi-lateral exchange of samples for analysis as an alternative. In Asia, the Forum for Nuclear Cooperation in Asia (FNCA) has an activity in which interlaboratory comparison testing is organized amongst NAA laboratories in FNCA member states [18]. Similar activities may be initiated amongst regional research reactor networks.

In 2020, the COVID-19 pandemic interrupted this activity. The IAEA intends to continue organizing proficiency testing exercises for NAA laboratories from 2021 onwards, combining it with contributions from laboratories operating nuclear-related analytical techniques for measurement of elements, such as based on X-ray fluorescence. As in the past, the PTs will be followed-up by planned feedback workshops for elaboration on results and further improvement of the measurement processes.