A Morphological Classification Model to Identify Unresolved PanSTARRS1 Sources. II. Update to the PS1 Point Source Catalog

PS1 conducted a five filter (${g}_{\mathrm{PS}1}$, ${r}_{\mathrm{PS}1}$, ${i}_{\mathrm{PS}1}$, ${z}_{\mathrm{PS}1}$, ${y}_{\mathrm{PS}1}$) time-domain survey covering ∼3/4 of the sky (Chambers et al. 2016). PS1 provides three different types of photometric measurements: there are mean flux measurements from the individual PS1 exposures of each field, there are stack flux measurements from the deeper stack images that co-add individual exposures, and there are forced-flux measurements that measure the flux in individual exposures at the location of all sources detected in the stack images. The mean photometry is limited by the depth of the individual exposures, while the stack photometry has a difficult to model point-spread function (PSF) because images must be warped before they can be co-added. The forced-flux measurements provide an intermediate compromise as they are deeper than the mean flux measurements, while in principle having a more stable PSF than the stack images.

Tachibana & Miller (2018) show that the stack photometry works best when morphologically classifying resolved, extended sources and unresolved point sources. The methodology that we adopt here is extremely similar to Tachibana & Miller (2018), but we instead use PS1 forced photometry to classify sources that do not have suitable stack photometry. The forced-photometry-based model leads to slightly lower quality classifications (see Section 5).

As a training set for the model, we use deep observations of the COSMOS field from the Hubble Space Telescope (HST). The superior resolution of HST enables reliable morphological classifications for sources as faint as ∼25 mag (Leauthaud et al. 2007). There are 80,867 bright HST sources from Leauthaud et al. (2007) that have PS1 counterparts (within a 1'' match radius; see Tachibana & Miller 2018) in the PS1 ForcedMeanObject table (see Section 3.1). Of those, the 47,825 PS1 sources with ${\mathtt{nDetections}}\geqslant 1$ are adopted as the training set for our model. This training set is ∼1.6% larger than the one used in Tachibana & Miller (2018) because more HST/COSMOS sources are "detected" in PS1 forced photometry. ⁶

Regardless of the choice of algorithm, the basic goal of a machine learning model is to build a map between source features, numerical and/or categorical properties that can be measured for an individual source, and labels, the target output, often a classification, of the model. This mapping is learned via a training set, a subset of the data with known labels, after which the model can classify any source based on its features.

Tachibana & Miller (2018) introduced the concept of "white flux" features, whereby measurements in the five individual PS1 filters were summed, via a weighted mean, to produce a "total" flux or shape measurement across all filters. ⁷ Machine learning models are limited by their training sets: there is no guarantee that their empirical mapping will correctly extend beyond the boundaries enclosed by the training set. Given the significant systematic uncertainties associated with Galactic reddening, and the tendency for spectroscopic samples, which are typically used to define training sets, to be biased in their target selection (see e.g., Miller et al. 2017), the motivation for "white flux" features becomes clear: they reduce potential biases in the final classifications due to selection effects in how the training set sources were targeted. Therefore, as in Tachibana & Miller (2018), we use "white flux" features in this study.

The PS1 StackObjectAttributes table provides both flux and shape (e.g., second moment of the radiation intensity) measurements in each of the five PS1 filters, whereas the PS1 ForcedMeanObject table only provides flux measurements. ⁸

To create the feature set for our machine learning model, we create "white flux" features for the six different flux measurements available in the ForcedMeanObject table ⁹ (FPSFFlux, FKronFlux, FApFlux, FmeanflxR5, FmeanflxR6, FmeanflxR7), as well as the E1 and E2 measurements, which represent the mean polarization parameters from Kaiser et al. (1995). We use flux ratios, rather than the raw flux measurements, which provide morphological classifications that are independent of S/N (Lupton et al. 2001).

Our final model includes nine features, five flux ratios:

$$\begin{eqnarray*}&&\begin{array}{l}{\mathtt{whiteFPSFApRatio}}\quad =\ \displaystyle \frac{{\mathtt{whiteFPSFlux}}}{{\mathtt{whiteFApFlux}}},\\ {\mathtt{whiteFPSFKronRatio}}\\ \quad =\ \displaystyle \frac{{\mathtt{whiteFPSFlux}}}{{\mathtt{whiteFKronFlux}}},\\ {\mathtt{whiteFPSFFmeanflxR}}{\mathtt{5}}{\mathtt{Ratio}}\\ \quad =\ \displaystyle \frac{{\mathtt{whiteFPSFlux}}}{{\mathtt{whiteFmeanflxR}}{\mathtt{5}}{\mathtt{Flux}}},\\ {\mathtt{whiteFPSFFmeanflxR}}{\mathtt{6}}{\mathtt{Ratio}}\\ \quad =\ \displaystyle \frac{{\mathtt{whiteFPSFlux}}}{{\mathtt{whiteFmeanflxR}}{\mathtt{6}}{\mathtt{Flux}}},\\ {\mathtt{whiteFPSFFmeanflxR}}{\mathtt{7}}{\mathtt{Ratio}}\\ \quad =\ \displaystyle \frac{{\mathtt{whiteFPSFlux}}}{{\mathtt{whiteFmeanflxR}}{\mathtt{7}}{\mathtt{Flux}}},\end{array}\end{eqnarray*}$$

the white polarization parameters: whiteE1 and whiteE2, and two "simple" distance measures: whiteFPSFApDist and whiteFPSFKronDist (see Section 3.2). The distribution of these features for stars and galaxies in the training set is shown in Figures 1–3.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Figure 1 shows that whiteFPSFApRatio is the most useful feature, aside from the "simple" features, to separate resolved and unresolved sources. This intuitively makes sense as PS1 ApFlux measurements are matched to the seeing, whereas the R5flx, R6flx, R7flx measurements use fixed aperture sizes. With multiple images taken under different observing conditions contributing to the final forced flux measurements, fixed aperture measurements should be more noisy.

Tachibana & Miller (2018) introduced a "simple" model to classify sources based solely on their measured whitePSFFlux and whiteKronFlux. The model was inspired by the use of flux ratios, which have been shown to provide a good discriminant between resolved and unresolved sources (e.g., the SDSS morphological CLASS parameter; Lupton et al. 2001). At moderate to low S/N, however, flux ratios no longer provide accurate classifications (see e.g., Figure 1). The simple model from Tachibana & Miller (2018) leverages this fact by measuring the distance of each source from a line drawn in the whitePSFFlux–whiteKronFlux plane. Unlike a flux ratio, the simple model preserves information about the S/N, meaning sources with large absolute distances from the dividing line can be classified with greater confidence.

Following from Equation (3) in Tachibana & Miller (2018), "simple" features can be calculated as:

$$\begin{eqnarray}&&{\mathtt{whiteF}}{\mathtt{1}}{\mathtt{F}}{\mathtt{2}}{\mathtt{Dist}}(a)=\displaystyle \frac{{\mathtt{whiteF}}{\mathtt{1}}-a\times {\mathtt{whiteF}}{\mathtt{2}}}{\sqrt{1+{a}^{2}}},\end{eqnarray} \tag{ 1 }$$

where whiteF1 and whiteF2 are the "white flux" measurements introduced in Section 3.1 (e.g., whiteFKronFlux), a is the slope of the line in the whiteF1–whiteF2 plane, and whiteF1F2Dist is the orthogonal distance of a source from the line (sources above the line have positive values). For this study we construct two simple features for inclusion in our machine learning model: whiteFPSFFKronDist and whiteFPSFFApDist.

We determine the optimal value of a for the simple features via cross validation. We find a = 0.7512 for the whiteFPSFFKronDist feature and a = 0.7784 for the whiteFPSFFApDist feature maximizes the FoM (see Section 4). Empirically whiteFPSFFApDist is better at separating resolved and unresolved sources than whiteFPSFFKronDist, and therefore the "simple" model, discussed below, is based on whiteFPSFFApDist. The whiteFPSFFApDist and whiteFPSFFKronDist distribution of resolved and unresolved sources is shown in Figures 2 and 3, respectively.

Our aim is to maximize the FoM of the RF model. We show receiver operating characteristic (ROC) curves of the RF, simple, and PS1 ¹² models in Figure 4. From Figure 4, it is clear that the RF and simple models greatly outperform the PS1 model. Furthermore, while the gains are modest, the inclusion of all the "white flux" features and use of machine learning is justified as the RF model produces a higher FoM than the simple model.

Zoom In Zoom Out Reset image size

The FoM of each of the three models is summarized in Table 1. In addition to providing the largest FoM, the RF model is also the most accurate and it has the largest area under the ROC curve (ROC AUC). We robustly conclude that, of the models considered here, the RF model is best. Comparing with Table 1 in Tachibana & Miller (2018), we find that the forced-photometry features derived in this study do not provide the same discriminating power as the PS1 stack-photometry features used in Tachibana & Miller (2018). Our new model performs ∼7% worse than the one in Tachibana & Miller (2018). In Section 6, we argue that this slight reduction in performance is more than offset by the ∼144 million additional sources that are now classified using the forced-photometry features.

We show the CV accuracy of the RF, simple, and PS1 models as a function of whiteFKronMag in Figure 5. As in Tachibana & Miller (2018), we find that the RF model provides more accurate classifications than the alternatives.

Zoom In Zoom Out Reset image size

The accuracy of each model shown in Figure 5 decreases for lower S/N sources. The accuracy curve for the RF and simple models feature a slight departure from expectation in that they do not decrease much from 22 to 24 mag. This quasi-plateau in the model accuracy can be understood as the result of two components of the training set: (i) unresolved sources completely dominate the source counts at these magnitudes, and (ii) the well-defined locus of unresolved sources in the training set (see Figure 1) becomes heavily blended with the resolved source population at these brightness levels. Taken together the model will be biased toward classifying all faint sources as resolved, despite the fact that we do not explicitly include flux measurements in the feature set. With 88.5% of the ${\mathtt{whiteFKronMag}}\gt 22.5$ mag training set sources being unresolved, a quasi-plateau in accuracy of ∼88% makes sense. This is confirmed in the bottom panel of Figure 5, which shows the RF model true positive rate (TPR) for both resolved and unresolved sources as a function of whiteFKronMag. A near 100% TPR for faint resolved sources combined with a few correctly classified unresolved sources leads to the observed quasi-plateau in Figure 5.

With a new RF model in hand, we can now provide morphological classifications for the PS1 sources that are currently missing from the PS1 PSC. Of the ∼426 million "missing" sources, ∼144 million have PS1 DR2 ForcedMeanObject photometry that pass our detection criteria (see Appendix for more details). A histogram showing the distribution of the RF classification score for these newly classified sources is shown in Figure 6.

Zoom In Zoom Out Reset image size

Figure 6 shows that there are relatively few high-confidence classifications (i.e., very likely extended sources with RF score ≈0 or very likely point sources with RF score ≈1 among the "missing" sources. Figure 6 also reveals the likely explanation for this outcome: the vast majority of the newly classified sources are in the Galactic plane. Of the ∼144 million newly classified sources, ∼57% have galactic latitude $| b| \lt 5$ deg, while >95% are in the Galactic plane ($| b| \lt 15$ deg). The HST COSMOS field, from which we derive our training set, has b ≈ 42 deg and as a result includes very few stellar blends, which are common at low galactic latitudes. The PS1 PSC also has significantly lower confidence classifications in the Galactic plane (see Figure 8 in Tachibana & Miller 2018). That these sources were not "detected" in the PS1 stack images also suggests that it is difficult to make reliable photometric measurements using the PS1 data, which could also contribute to the lower confidence classifications. We use the third data release from the space-based Gaia telescope (Perryman et al. 2001) to improve this situation by classifying many of these ambiguous sources as stars via parallax and proper motion measurements (Section 6.1).

Ultimately, this update to the PS1 PSC has identified 17,945,494 likely point sources using the optimized threshold from Tachibana & Miller (2018, RF score ≥0.83). While this number is small compared to the ∼734 million point sources in the original PS1 PSC, these ∼18 million newly identified point sources would otherwise pass filters looking for extragalactic transients in the ZTF alert stream. Their removal will reduce the number of false positive transient candidates.

The Gaia Early Data Release 3 includes high-precision astrometric measurements collected over a 34 months timespan for ∼1.8 billion sources (Gaia Collaboration et al. 2020). Within ZTF, the PS1 PSC is primarily used to identify likely stars (i.e., point sources) and remove them from filters searching for extragalactic transients. To that end, we can supplement the RF classifications described in Section 5.2 with Gaia stars, which are identified via high-significance parallax and proper motion detections.

A common threshold for determining "high-significance" is ${\rm{S}}/{\rm{N}}\geqslant 5$, which in the case of Gaussian uncertainties corresponds to a ∼3$\times {10}^{-7}$ probability that the observed signal is the result of noise. We can therefore select stars from Gaia sources with high S/N parallax or proper motion measurements. ¹³ We adopt conservative significance thresholds because the formal uncertainties from Gaia are slightly underestimated (Fabricius et al. 2020) and because most of the "missing" sources in the ZTF Stars table are in the Galactic Plane (e.g., Figure 6). Fabricius et al. (2020); estimate that Gaia parallax measurements underestimate the uncertainties by a much as ∼60% in crowded regions. Similarly, proper motions are found to be underestimated by as much ∼80% in crowded regions (Fabricius et al. 2020). We therefore only consider Gaia sources with a parallax ${\rm{S}}/{\rm{N}}\geqslant 8$ or a total proper motion ${\rm{S}}/{\rm{N}}\geqslant 9$ to be stars.

Using the ESA Gaia archive ¹⁴ we find there are 18,658,572 sources with either a high-significance parallax or proper motion detection in the ZTF Stars table that lack a classification in the original PS1 PSC. For these sources (11,427,503 of which have RF scores from Section 5.2) we update their scores to 1 in the ZTF Stars table. This effectively excludes each of these sources from filters designed to find extragalactic transients in the ZTF alert stream.

Moving forward, ZTF alert packets now include ∼152 million additional classifications (∼133.6 million RF classifications from Section 5.2, and ∼18.6 million from Section 6.1. The addition of these new classifications to the ZTF AVRO packets should not affect existing alert-stream filters, as we describe below.

While a one-to-one mapping of point-source classification scores cannot be made between Tachibana & Miller (2018) and this study, the similarity between the two methodologies leads to classifications that are highly similar. Table 2 summarizes the TPR and FPR for different classification thresholds using the model from Tachibana & Miller (2018) and the RF model created in this study. The PS1 stack photometry used in Tachibana & Miller (2018) consistently produces a higher TPR, by ∼6%–8%, than the PS1 forced photometry. The PS1 forced photometry used in this study does have a lower FPR than Tachibana & Miller (2018) for all but the most liberal point-source classification cuts. Thus, applying classification cuts developed for the original PS1 PSC will ultimately lead to a higher TPR, as previously unclassified point sources can now be removed from the stream, without experiencing an overall increase in the FPR. As a result, we conclude that the vast majority of users will not experience any significant change in the results to their filters, aside from a slight reduction in false negatives (stars that are classified as galaxies), following the update to the ZTF Stars table.

Table 2. TPR and FPR for TM18 Thresholds

Catalog	Threshold	0.829	0.724	0.597	0.397	0.224
TM18	TPR	0.734	0.792	0.843	0.0904	0.947
	FPR	0.005	0.01	0.02	0.05	0.1
This work	TPR	${0.684}_{-0.005}^{+0.005}$	${0.730}_{-0.005}^{+0.004}$	${0.772}_{-0.004}^{+0.004}$	${0.833}_{-0.003}^{+0.004}$	${0.902}_{-0.004}^{+0.004}$
	FPR	${0.005}_{-0.000}^{+0.001}$	${0.009}_{-0.001}^{+0.001}$	${0.016}_{-0.001}^{+0.001}$	${0.041}_{-0.001}^{+0.002}$	${0.111}_{-0.003}^{+0.002}$

Note. The table reports the TPR and FPR for different classification thresholds given in Table 3 in Tachibana & Miller (2018). To estimate the TPR and FPR we perform 10-fold CV on the entire training set, but only include sources with ${\mathtt{nDetections}}\geqslant 3$ in the final TPR and FPR calculations. The first row (TM18) summarizes the results from Tachibana & Miller (2018), while the second row uses the RF model from this study. The reported uncertainties represent the central 90% interval from 100 bootstrap resamples of the training set.

Download table as:  ASCIITypeset image