Comparison of conventional and Si-photomultiplier-based PET systems for image quality and diagnostic performance

Abstract

Background

A new generation of positron emission tomography with computed tomography (PET-CT) was recently introduced using silicon (Si) photomultiplier (PM)-based technology. Our aim was to compare the image quality and diagnostic performance of a SiPM-based PET-CT (Discovery MI; GE Healthcare, Milwaukee, WI, USA) with a time-of-flight PET-CT scanner with a conventional PM detector (Gemini TF; Philips Healthcare, Cleveland, OH, USA), including reconstruction algorithms per vendor’s recommendations.

Methods

Imaging of the National Electrical Manufacturers Association IEC body phantom and 16 patients was carried out using 1.5 min/bed for the Discovery MI PET-CT and 2 min/bed for the Gemini TF PET-CT. Images were analysed for recovery coefficients for the phantom, signal-to-noise ratio in the liver, standardized uptake values (SUV) in lesions, number of lesions and metabolic TNM classifications in patients.

Results

In phantom, the correct (> 90%) activity level was measured for spheres ≥17 mm for Discovery MI, whereas the Gemini TF reached a correct measured activity level for the 37-mm sphere. In patient studies, metabolic TNM classification was worse using images obtained from the Discovery MI compared those obtained from the Gemini TF in 4 of 15 patients. A trend toward more malignant, inflammatory and unclear lesions was found using images acquired with the Discovery MI compared with the Gemini TF, but this was not statistically significant. Lesion-to-blood-pool SUV ratios were significantly higher in images from the Discovery MI compared with the Gemini TF for lesions smaller than 1 cm (p < 0.001), but this was not the case for larger lesions (p = 0.053). The signal-to-noise ratio in the liver was similar between platforms (p = 0.52). Also, shorter acquisition times were possible using the Discovery MI, with preserved signal-to-noise ratio in the liver.

Conclusions

Image quality was better with Discovery MI compared to conventional Gemini TF. Although no gold standard was available, the results indicate that the new PET-CT generation will provide potentially better diagnostic performance.

Background

Positron emission tomography with computed tomography (PET-CT) is a well-established and fast-growing imaging modality, mainly used in oncology [1]. Over the years, improvements in detector design as well as the implementation of time-of-flight technology and better reconstruction methods have significantly improved image quality and introduced the possibility of reducing the amount of activity administered and/or scanning time [2]. A new generation of PET-CT scanners has recently been introduced, and they use silicon (Si) photomultiplier (PM)-based technology. This technology has the potential to increase the detection of pathology, primarily through greater sensitivity. It is hoped that this improved detectability will enable earlier detection of pathologies, including metastatic spread. Thus far, this remains to be confirmed in patient studies. Using the National Electrical Manufacturers Association (NEMA) NU-22012 standard [3], Hsu et al. [4] found that SiPM-based PET cameras have greater sensitivity and time resolution compared to conventional PM-based PET cameras. They also presented a clinical case where metastases were detected using the SiPM-based PET camera but not using a conventional PM-based PET camera. Two other studies comparing SiPM-based PET-CT to conventional PET-CT found the SiPM-based PET-CT performed better and detected a greater number of lesions [5, 6].

The aim of this study was to compare the quality and diagnostic performance of images obtained from a novel SiPM-based PET system (Discovery MI; GE Healthcare, Milwaukee, WI, USA), using the Bayesian penalized-likelihood reconstruction algorithm (Q.Clear), with those obtained from a conventional PM-based PET system with time-of-flight (Gemini TF; Philips Healthcare, Cleveland, OH, USA), using the line-of-response row-action maximum-likelihood algorithm, in phantom and patients undergoing clinical PET-CT scanning with ¹⁸F-FDG. Thus, images from two PET-CT cameras with reconstruction algorithms per vendor’s recommendations were compared; i.e. clinically relevant protocols were evaluated.

Methods

The PET-CT systems

The imaging of the phantom and the patients was carried out on two different PET-CT systems, a GE Discovery MI installed 2017 and a Philips Gemini TF installed 2007, see Table 1 for specifications. Both PET-CT scanners are cross-calibrated to the activity meter, and the calibration is validated monthly in a standardized uptake value (SUV) control of phantoms.

Table 1 Characteristics of the Discovery MI and Gemini TF PET-CT systems

Phantom studies

The NEMA IEC Body Phantom (Data Spectrum, Durham, NC, USA) with fillable spheres (10, 13, 17, 22, 28 and 37 mm diameter) and a cylindrical lung insert was used. Its background volume was filled with an ¹⁸F-FDG concentration of 4.7 ± 0.1 kBq/mL for the Discovery MI and 4.9 ± 0.1 kBq/mL for the Gemini TF. To yield a sphere-to-background activity ratio of four, activity concentrations of 18.8 ± 0.4 kBq/mL and 18.7 ± 0.4 kBq/mL, respectively, were used for the spheres. The phantom was scanned in one bed position using the clinical parameters on each camera, which were 1.5 min/bed position, for the Discovery MI and 2 min/bed position, for the Gemini TF. Acquisition was carried out in list mode with time-of-flight enabled for both systems. No motion-correction was performed. All CT imaging was performed using a low-dose CT protocol at 120 kV, 30–160 mA and a noise index of 45 for the Discovery MI and at 120 kV and 50 mA without tube current modulation for the Gemini TF. Image reconstructions were performed according to routine clinical practice with reconstruction parameters to obtain the best clinical image per system. For the Discovery MI, reconstruction was performed using a Bayesian penalized-likelihood reconstruction algorithm (Q.Clear) [7] with a beta value of 550, using time of flight and the point spread function modelling for recovery of the resolution. For the Gemini TF, the line-of-response row-action maximum-likelihood algorithm method, the so-called BLOB-OS-TOF was used, with time of flight but without the point spread function over 3 iterations, 33 subsets and a 5-mm Gaussian post-filter.

In the image corresponding to the central slice of spheres, regions of interests (ROIs) were drawn along the inside borders of the spheres in the CT image, and this was copied onto the PET images. Mean activity concentrations in spheres and backgrounds were then measured. The recovery coefficient (RC) was calculated as the ratio between the measured and the true activity concentration.

Patient studies

Patients

Patients aged 18 years or more who were referred for clinical ¹⁸F-FDG-PET-CT to Skåne University Hospital, Malmö between May 82,017 and May 232,017 were invited to participate in the study. A total of 16 patients were included. This study was approved by the Regional Ethical Review Board (#2016/417) and the Radiation Protection Committee (#SSFo2016–018) and was performed in accordance with the Declaration of Helsinki. All patients gave written consent.

Imaging

All patients underwent a single intravenous injection of ¹⁸F-FDG (4 MBq/kg) after fasting for at least 4 h and a dual-imaging protocol on the Discovery MI and Gemini TF PET-CT scanners. Of the 16 patients, nine were first imaged on the Gemini TF, and seven were first imaged on the Discovery MI. For the first acquisition, we aimed for a 60-min accumulation time, as per standard practice in our clinic. Images were acquired from the inguinal region to the base of the skull (1.5 min per bed position for the Discovery MI and 2 min per bed position for the Gemini TF). The images were reconstructed as described above. No gating for respiratory motion was applied.

CT imaging was acquired for attenuation correction and anatomic correlation of the PET images. According to our clinical routine, a diagnostic CT with intravenous contrast was performed if no previous diagnostic CT had been performed within 4 to 6 weeks. In 9 patients, a diagnostic CT was performed with the first PET acquisition, and a low-dose CT was performed with the second PET acquisition. In two patients, the opposite order was applied. In the remaining five patients, low-dose CT was performed for both PET acquisitions. For diagnostic CTs, tube current modulation was applied (the adaptive statistical iterative reconstruction V-technique, ASiR-V for Discovery MI and DoseRight for Gemini TF), adjusting the tube current for each individual and thus optimising the radiation dose. A tube voltage of 100 kV was used for the Discovery MI and 120 kV for the Gemini TF. The low-dose CT was acquired using the same tube current and voltage as described for the phantom CT acquisitions.

Image analysis

Image quality

The signal-to-noise ratio (SNR) was calculated from ROIs drawn in liver transaxial images. A 4 cm² ROI was drawn in liver segment 5/6 on images acquired from the Discovery MI, and it was copied to the images acquired from the Gemini TF. The ROIs were drawn in three subsequent transaxial slices, and the measurements were averaged. None of the ROIs were placed where liver metastasis was seen. The SNR was calculated by dividing the SUV_mean in the ROI by its standard deviation. The SUVs and lesion-to-blood-pool (blood pool measured in the left atria) SUV ratios were calculated by an experienced nuclear medicine physician. A maximum of six ¹⁸F-FDG-avid lesions were selected for each patient (the same for both camera systems). If available, malignant lesions were included; otherwise lesions interpreted as inflammatory or unclear were included. Lesions were chosen so that a variety of sizes (< 1 cm and ≥ 1 cm on the short axis measured on transversal CT images) were included. If possible, lesions from a variety of body regions were chosen. The lesion-to-blood-pool ratios were calculated as: lesion SUV_max/blood pool SUV_mean.

Image interpretation

All images were reviewed on an Extended Brilliance Workspace workstation (Philips Healthcare). Three experienced nuclear medicine physicians together reviewed the two PET datasets. Images were analysed for the number of pathologic lesions and were classified as “malignant/probably malignant lesion”, “inflammatory/probably inflammatory lesion” or “unclear lesion”. Furthermore, a simplified metabolic TNM classification (if applicable) was performed. It was defined as follows: was a primary tumour present or not (T+ or T-), did it spread to lymph nodes or not (N+ or N-) and were distant metastases present or not (M+ or M-). The presence of T+, N+ or M+ was considered a worse stage than T-, N- or M-. Patient images were anonymized and randomized, then given to the experts for interpretation. Image sets were separated by camera, and those acquired using the Gemini TF were interpreted first. The physicians were provided with basic patient information (i.e., sex, age, indication for the examination, and the clinically-written CT report). Where only a low-dose CT was performed together with the PET acquisition, the diagnostic CT performed on the other scanner (or, if a separate diagnostic CT was performed within 4–6 weeks before the PET-CT, according to routine clinical practice), was provided to the interpreting physicians (see Imaging section).

Statistics

Differences in SNR in the liver, number of lesions and SUV parameters were assessed using paired t-tests. Differences in metabolic TNM classification were calculated as 95% confidence intervals (CIs) of a proportion using the modified Wald method. Statistical significance was recognized at p ≤ 0.05 unless otherwise specified. Bonferroni adjustments were used to account for multiple comparisons for SUV lesion parameters.

Results

Phantom studies

Figure 1 shows images obtained from the phantom on both systems. Measured mean activity concentration was underestimated by both for the 10-mm diameter spheres. The Discovery MI measured the activity level accurately compared to the activity meter (> 90%) for spheres 17 mm or greater in diameter. As seen in Fig. 2, the underestimation was comparatively larger when using the Gemini TF with spheres measuring 13, 17, 22 or 28 mm in diameter. The Gemini TF reached an activity level close to that measured by the activity meter (> 90%) for the 37-mm spheres. Table 2 shows the activity levels measured for the spheres and backgrounds. The SNRs in the backgrounds were 11.7 and 14.9 for the Discovery MI and Gemini TF, respectively.

Fig. 1

Transversal images of the phantom obtained from a) Discovery MI and b) Gemini TF with a sphere-to-background activity ratio of 4:1. Image acquisitions and reconstructions were performed as in the patient study

Fig. 2

Recovery coefficients for two scanners across various sphere diameters in the phantom with a sphere-to-background activity ratio of 4:1. Scanning acquisition and image reconstruction parameters identical to those used for a clinical patient. Results based on the mean activity concentration achieved from regions of interest drawn on the CT image then copied to the PET image

Table 2 Measured activity concentration in images from the Discovery MI and the Gemini TF compared to the true activity concentration (measured by ionization chamber)

Patient studies

Patients

Sixteen (16) patients were included. Patient characteristics and their indications for the PET-CT are shown in Table 3. Mean accumulation time for the nine patients first imaged on the Gemini TF was 66 ± 7 min for the Gemini TF and 103 ± 10 min for the Discovery MI. For those imaged in the reverse order, mean accumulation time was 67 ± 8 min for the Discovery MI and 98 ± 17 min for the Gemini TF. Overall, mean accumulation time was 91 ± 17 min for images obtained with the Discovery MI and 76 ± 22 min for images obtained with the Gemini TF (p = 0.04).

Table 3 Patient characteristics

Image quality and lesion analysis

The mean SNR in the liver was 11.9 ± 3.2 and 12.7 ± 2.7 for the Discovery MI and Gemini TF, respectively. The difference was not statistically significant (p = 0.52). SUV_mean in the liver was 2.4 for both systems.

In total, 68 lesions were analysed regarding SUV, 32 of which measured smaller than 1 cm. Lesion SUV_max was 8.3 ± 6.5 for images from the Discovery MI and 5.2 ± 4.5 for images from the Gemini TF (p < 0.001). The SUV_mean in the blood pool was 1.7 ± 0.4 in images from the Discovery MI and 1.5 ± 0.3 in images from the Gemini TF (p = 0.02). The lesion-to-blood-pool ratio was 4.9 ± 4.0 for images from the Discovery MI and 3.7 ± 3.4 for images from the Gemini TF (p < 0.001). Details of lesion SUV parameters and subgroup analysis for all lesions are shown in Table 4. All absolute SUV_max values were greater in lesions found using the Discovery MI compared to those found using the Gemini TF, with one exception. For patients first imaged on the Discovery MI, the difference was 2.3 ± 1.9 (n = 30), and for those first imaged on the Gemini TF, it was 4.1 ± 3.5 (n = 38; p = 0.02).

Table 4 Lesion SUV_max parameters (mean ± standard deviation)

Image interpretation

The total number of malignant/suspected malignant lesions was 32 for images obtained from the Discovery MI and 24 for images obtained from the Gemini TF (p = 0.16). The number of inflammatory/suspected inflammatory lesions was 55 for images obtained from the Discovery MI and 29 for images obtained from the Gemini TF (p = 0.09). The number of lesions of unclear significance was 38 for the Discovery MI and 24 for the Gemini TF (p = 0.27). Table 5 shows the number of lesions found for each patient. Metabolic TNM classification was performed for 15 patients (not for the patient with sarcoidosis). The presence/absence of primary tumour (T) differed in one of the patients (“present” for images from the Discovery MI and “uncertain” for images from the Gemini TF). The presence/absence of spread to lymph nodes (N) was different in one patient (“absent” for images from the Gemini TF and “uncertain” for images from the Discovery MI). The presence/absence of distant metastases (M) was different in two patients (“absent” for images from the Gemini TF and “present” for images from the Discovery MI for both patients). For all other patients, T, N and M classifications were identical for both image sets. Overall, metabolic TNM classification differed in four patients (27%; 95% CI, 10–52%) between the two systems. Table 6 shows the details of the metabolic TNM classifications. Figure 3 shows Patient 14, who was interpreted as having a primary tumour in the right lung when images from the Discovery MI were interpreted (T+), but not when images from the Gemini TF were interpreted (T-). Figure 4 shows Patient 9, where the hyper-metabolism in carcinomatosis is seen more clearly in the Discovery MI images compared to the Gemini TF images.

Table 5 Number of lesions for various patients

Table 6 Presence (+) or absence (−) of primary tumour (T), lymph node metastases (N) and distant metastases (M)

Fig. 3

a) A transversal image from the Gemini TF. b) The same image from the Discovery MI and c) the corresponding CT image. The arrow indicates a tumour in the right lung, which was interpreted as malignant in b

Fig. 4

a) A transversal image from the Gemini TF. b) The same image from the Discovery MI and c) the corresponding CT image. The arrow indicates carcinomatosis, more clearly visible in b

Discussion

The primary finding of this study was that metabolic TNM classifications indicated tumours were at a worse stage (presence of primary tumour, spread to lymph nodes or distant metastases) for images obtained from the Discovery MI compared to those obtained from the Gemini TF in 27% of patients. Also, a trend toward a greater number of malignant, inflammatory and unclear lesions was found with images obtained with the Discovery MI compared to the Gemini TF (although not statistically significant). Also, lesion-to-blood-pool SUV ratios were significantly greater for Discovery MI images compared to Gemini TF images for lesions smaller than 1 cm. Phantom studies showed better image quality for the Discovery MI.

It is clear from analyses of patient lesions and from the phantom analysis that lesions, especially smaller ones, have higher SUVs and are more visible on images obtained from the Discovery MI compared to those from the Gemini TF. However, SUV measurements can be influenced by many factors, including accumulation time [8, 9], patient characteristics and reconstruction method [7, 10,11,12]. Although SUVs were higher in all but one lesion for images from the Discovery MI, a significantly greater lesion-to-blood-pool SUV ratio was found for lesions smaller than 1 cm. Therefore, the greater SUV is likely explained by the greater sensitivity and different reconstruction algorithm of the Discovery MI system rather than longer accumulation times. The new reconstruction algorithm used with the Discovery MI will decrease the effect of partial volume, particularly for small lesions, due to the point spread function in the Bayesian penalized-likelihood reconstruction algorithm. Using the Gemini TF, this partial volume correction was not possible, leading to a more pronounced underestimation of SUVs in small lesions. Also, the smaller voxel size used by the Discovery MI (2.7 × 2.7 × 2.8 mm) compared to the Gemini TF (4 × 4 × 4 mm) contributes to increased SUVs found using the Discovery MI [13]. A different SUV could also be obtained when using the Q. Clear algorithm because the noise-penalizing determining factor (beta) affects SUV. Thus, our findings depend on both hardware (different generation of PET-CT cameras) and software (different reconstruction methods). The aim of this study was to compare the clinical performances of the two systems. Therefore, the clinical protocols recommended by the vendors were used rather than the same reconstruction method for both.

The greater sensitivity of the Discovery MI permitted shorter times per bed position (1.5 min compared to 2 min for the Gemini TF) with preserved SNRs in the liver for a given dose (4 MBq/kg). The greater axial field of view (20 cm) and the short overlap (24%) permit faster image acquisition. Generally, our experience is that the total PET acquisition time is half that for the Gemini TF. A short acquisition time potentially reduces problems with patient motion and bladder filling and would allow more patients to be examined per day.

It is hoped that the greater sensitivity of SiPM PET-CT and improved reconstruction algorithms will lead to increased detection of pathology, such as earlier detection of small metastases. We found that 27% of patients had different (worse) metabolic TNM classifications using images acquired from the Discovery MI compared to those acquired from the Gemini TF. Unfortunately, we do not know the patients’ true TNM classifications, but our results indicate that more primary tumours, lymph nodes or distant metastases can be detected using the new generation of PET-CT scanners. This agrees with our findings that more malignant and inflammatory lesions were detected (although the difference was not statistically significant, probably because of the limited number of patients included). Also, a greater number of uncertain lesions were found using the Discovery MI PET-CT system. This could be because the physicians interpreting the images were more familiar with images from the Gemini TF, but it could also be a result of the greater number of hypermetabolic lesions found using the novel PET systems. In subsequent studies, where a gold standard is available, it is important to assess both sensitivity and specificity for various diseases using the new generation of PET-CT.

Previous studies of SiPM-based PET platforms in patients have been published. Hsu et al. [4] used the same SiPM-based PET platform as in our study. They presented performance studies of the Discovery MI system and also included a patient case, comparing with a previous-generation PET-CT. The patient, who had melanoma, was scanned using a Discovery 690 PET-CT system (GE Healthcare) which was immediately followed by imaging using the Discovery MI. Several lesions visible only in the Discovery MI images were found, but no gold standard was used to assess true pathology. Nguyen et al. [6] compared the diagnostic performance of a SiPM-based PET prototype scanner with a conventional PM PET scanner using 21 patients who underwent clinical ¹⁸F-FDG PET-CT. Use of a Gemini TF was followed by use of the SiPM-based prototype, showing better overall image quality with the latter. Lesion SUV_max and lesion-to-blood-pool SUV ratios were significantly greater using the SiPM PET compared to conventional PET, and more so for lesions smaller than 1.5 cm. Baratto et al. [5] scanned 50 patients using a conventional PM-based PET-CT followed by a SiPM-based PET-CT, finding that using the latter, more lesions were found as well as higher values for SUV_max, lesion-to-blood-pool SUV ratio and liver SUV ratio. The findings of these studies agree with our findings. In all these studies, images were acquired using the conventional PM-based PET-CT first. In our study, we attempted to eliminate differences in accumulation time on the two camera systems by scanning about half the patients first using the Gemini TF PET-CT and half in the reverse order.

Additional studies are needed to establish the full potential value of the new generation of PET-CT scanners, but the results from our study, as well as previous studies, indicate that the increased lesion uptake in the new generation of PET-CT, thanks to improved hardware and reconstruction algorithms, can improve diagnostic performance. It is hoped that smaller lesions can be detected and that this will increase sensitivity and specificity for diagnosing various diseases, potentially improving patient management and outcomes.

Limitations

This study should be viewed in light of some limitations. First, a limited number of patients were included. Second, no gold standard, such as biopsy, or follow-up were performed so that the true number of lesions or true TNM classifications could be assessed. Third, some patients received a diagnostic CT (with intravenous contrast) for one of the PET-CT examinations, whereas others received a low-dose CT. The presence of intravenous contrast is known to slightly affect SUV [14]. Fourth, because the group who received CTs with the Gemini TF first was larger than the group receiving CTs with the Discovery MI first (by two patients), a significantly shorter accumulation time was seen in images obtained from the Gemini TF, which affects SUV and possibly image interpretation. Fifth, no correction for respiratory movement was applied.

Conclusions

In conclusion, TNM classification was worse in 4 of the 15 patients evaluated for TNM classification, and a trend toward more malignant, inflammatory and unclear lesions was found with images acquired with the Discovery MI compared to the Gemini TF. Also, lesion-to-blood-pool SUV ratios were significantly higher for the Discovery MI compared to the Gemini TF for lesions smaller than 1 cm. It was possible to use a shorter acquisition time for the Discovery MI with preserved SNRs in the liver. Better image quality was found in phantom. Although no gold standard was available, results indicate that the new generation of PET-CT scanners might provide better diagnostic performance, though further studies are needed.