Original contributionAnatomical accuracy of standard-practice tractography algorithms in the motor system - A histological validation in the squirrel monkey brain
Introduction
Diffusion MRI fiber tractography is widely used to probe the structural connectivity of the brain, with a range of applications in both clinical and basic neuroscience [1,2]. However, these techniques are subject to a number of serious pitfalls and limitations which may limit the anatomical accuracy of the reconstructed pathways [3,4]. In addition, the large number of diffusion reconstruction algorithms and tracking strategies that exist are likely to result in different “tracts”, with varying levels of accuracy. As utilization of fiber tractography continually increases, it is necessary to validate these techniques in order to gain insight into the conditions under which they succeed, and more importantly, where they fail.
One approach to validation is through classical tracer injection techniques in animal models, followed by histological analysis to defined the “ground truth” pathways for subsequent comparisons with diffusion tractography. Traditionally, validating the faithfulness of tractography against tracers takes one of two forms. First, some metric of spatial overlap of the tract versus the tracer can be computed, which evaluates the overall layout or spatial extent of the tract. Second, many studies evaluate connectivity measures, disregarding how tracts reach their destinations, with a focus on the strength of the connections between different regions of the brain.
Connection strengths estimated from tractography have been compared with invasive tracer data accumulated in existing atlases or databases, for example the Markov-Kennedy [5] or CoCoMac [6] databases for the macaque, or the Allen Brain Atlas for the mouse [7]. These studies have provided encouraging results, finding moderate to high positive correlations between tractography and connection strengths [[8], [9], [10]], suggesting that the number of reconstructed streamlines is correlated with the strength of connections between brain regions. However, tractography becomes less reliable for longer pathways [10], and results are heavily dependent on decisions made during the tracking process (i.e. the seeding strategy). The use of large-scale tracer databases has the advantage of assessing connectivity of a large number of pathways across many cortical areas, however, they have several disadvantages. Most notably, tracer injection and MRI are typically not employed on the same animal (with few exceptions [11]). Not only can pathway connection strength vary between animals, but variance in brain geometry between injected and scanned animals could lead to mismatches in identifying the location of the injection regions in the subject of interest, together compromising the fidelity of the “ground truth” to which tractography is being compared.
Alternatively, a number of studies have investigated the voxel-wise spatial overlap of histologically-defined white matter trajectories with those from tractography. Validating these measures gives confidence in the ability of tractography to segment specific white matter pathways (with subsequent analysis typically taking some quantitative measure along these pathways). For example, Schmahmann et al. [12] compare one implementation of tractography (diffusion spectrum imaging) to histological tracing and conclude that tractography is able to replicate the major features and geometrical organization of a number of association pathways. Improving upon this, in a series of studies on the macaque brain, Dauget et al. [13,14] register histological sections of labeled fiber tracts in 3D to diffusion tensor imaging (DTI) tractography data. They find a range in spatial agreement, with a range of Dice overlap coefficients (0.2–0.75) dependent on the pathway of interest and various tractography parameters, and note that DTI has difficulties when tracts cross or divide, an issue now referred to as the “crossing fiber” problem.
Building upon these studies, the goal of the present work is to systematically characterize the anatomical accuracy of diffusion fiber tractography – both the spatial extent and tract connections – and to do this on both the scale of individual voxels as well on a larger domain over anatomical regions of interest. To achieve this goal, we utilize the squirrel monkey brain, and compare tractography results directly to registered high-resolution tracer data from the same animal. We aim here to evaluate the algorithms most commonly employed in the literature (all of which are implemented in open-source software packages) in order to reveal the successes and shortcomings of the majority of studies utilizing diffusion tractography to date. In addition to measures of overlap and connectivity for each algorithm, we further assess the effects of user-defined algorithm choices (reconstruction algorithm, seeding strategy, tracking logic), distance from seed point, and effects of probabilistic thresholding on the fidelity of resulting tractograms. The focus of this work is on tractography of the pathways in the motor system. This is because the organization and anatomical connections of this system are well understood [15], and the motor system is a frequent target of tractography as it is particularly relevant for a variety of disabilities or pathologies including stroke [16,17], multiple sclerosis [18,19], Parkinson's disease [20,21], cerebral palsy [22,23], and tumor removal surgeries [24,25], among others. Herein, we investigate the spatial errors in these tractography algorithms, asking where in the brain these algorithms typically fail, and assess potential reasons for this failure.
Section snippets
Methods
All animal procedures were approved by the Vanderbilt University Animal Care and Use Committee. Fig. 1 shows the methodology pipeline used in this study. Briefly, biotinylated dextran amine (BDA), a histological tracer, was injected into the primary motor cortex (M1) of two squirrel monkey brains. Afterwards, diffusion MRI was acquired on the ex vivo brains and diffusion fiber tractography performed using 40 different algorithms and/or tracking settings. These 40 tractograms resulted in both
Histological results
The “ground truth” BDA connectivity of the M1 injection region is shown in Fig. 2, as both BDA density maps overlaid on MRI coronal slices (A,C) and as a binary map (B,D) in a tri-planar view for each subject. Most notably, the highest BDA densities occur in the cortex of the injection region, with dense projection fibers down the corticospinal tract (CST) traversing the genu of the internal capsule (IC) and the cerebral peduncles (CP). As expected based on existing literature, the M1 injection
Discussion
Diffusion MRI tractography is the only non-invasive method that offers the ability to map the structural connectivity of the human brain, and its application has been widely adopted in both small and large-scale studies over the last two decades in order to improve our understanding of normal brain development as well as complex brain disorders. However, the application of these methods is arguably racing ahead of our ability to understand the data and its limitations. It is critical that these
Conclusions
Diffusion tractography has seen widespread use for investigating the structural connectivity of the human brain. Despite known limitations of common methods, and a large number of advanced algorithms and reconstruction methods, most studies still implement common, open-source tractography methodologies. We found that none of these standard-practice algorithms is consistently successful at recovering the spatial extent of fiber pathways, or revealing region-to-region connectivity. The anatomical
Acknowledgements
This work was supported by the National Institute of Neurological Disorders and Stroke of the National Institutes of Health under award numbers RO1 NS058639 and S10 RR17799. Whole slide imaging was performed in the Digital Histology Shared Resource at Vanderbilt University Medical Center (www.mc.vanderbilt.edu/dhsr).
References (72)
- et al.
White matter integrity, fiber count, and other fallacies: the do's and don'ts of diffusion MRI
NeuroImage
(2013) - et al.
Comparison of fiber tracts derived from in-vivo DTI tractography with 3D histological neural tract tracer reconstruction on a macaque brain
NeuroImage
(2007) - et al.
Diffusion tensor imaging of the corticospinal tract and walking performance in multiple sclerosis
J Neurol Sci
(2016) - et al.
White matter fiber tracking in patients with space-occupying lesions of the brain: a new technique for neurosurgical planning?
NeuroImage
(2003) - et al.
Pathway tracing using biotinylated dextran amines
J Neurosci Methods
(2000) - et al.
Diffusion imaging of whole, post-mortem human brains on a clinical MRI scanner
NeuroImage
(2011) - et al.
Reproducibility and variation of diffusion measures in the squirrel monkey brain, in vivo and ex vivo
Magn Reson Imaging
(2017) - et al.
Fsl
NeuroImage
(2012) - et al.
MR diffusion tensor spectroscopy and imaging
Biophys J
(1994) - et al.
Probabilistic diffusion tractography with multiple fibre orientations: what can we gain?
NeuroImage
(2007)
Robust determination of the fibre orientation distribution in diffusion MRI: non-negativity constrained super-resolved spherical deconvolution
NeuroImage
High-resolution anatomy from in situ human brain
NeuroImage
Accuracy of image registration between MRI and light microscopy in the ex vivo brain
Magn Reson Imaging
Tractometer: towards validation of tractography pipelines
Med Image Anal
Diffusion tensor imaging of time-dependent axonal and myelin degradation after corpus callosotomy in epilepsy patients
NeuroImage
SIFT: spherical-deconvolution informed filtering of tractograms
NeuroImage
Reproducibility of quantitative tractography methods applied to cerebral white matter
NeuroImage
An error analysis of white matter tractography methods: synthetic diffusion tensor field simulations
NeuroImage
Quantitative evaluation of 10 tractography algorithms on a realistic diffusion MR phantom
NeuroImage
FIBRASCAN: a novel method for 3D white matter tract reconstruction in MR space from cadaveric dissection
NeuroImage
Validation of in vitro probabilistic tractography
NeuroImage
Comparison of 3D orientation distribution functions measured with confocal microscopy and diffusion MRI
NeuroImage
Histological validation of diffusion MRI fiber orientation distributions and dispersion
NeuroImage
Atlas of human brain connections
Just pretty pictures? What diffusion tractography can add in clinical neuroscience
Curr Opin Neurol
Twenty-five pitfalls in the analysis of diffusion MRI data
NMR Biomed
A weighted and directed interareal connectivity matrix for macaque cerebral cortex
Cereb Cortex
Advanced database methodology for the collation of connectivity data on the macaque brain (CoCoMac)
Philos Trans R Soc Lond Ser B Biol Sci
A mesoscale connectome of the mouse brain
Nature
Comparison of diffusion tractography and tract-tracing measures of connectivity strength in rhesus macaque connectome
Hum Brain Mapp
A diffusion MRI tractography connectome of the mouse brain and comparison with neuronal tracer data
Cereb Cortex
Using diffusion tractography to predict cortical connection strength and distance: a quantitative comparison with tracers in the monkey
J Neurosci
Validation of DTI tractography-based measures of primary motor area connectivity in the squirrel monkey brain
PLoS One
Association fibre pathways of the brain: parallel observations from diffusion spectrum imaging and autoradiography
Brain
3D histological reconstruction of fiber tracts and direct comparison with diffusion tensor MRI tractography
Med Image Comput Comput Assist Interv
Architectonics, somatotopic organization, and ipsilateral cortical connections of the primary motor area (M1) of owl monkeys
J Comp Neurol
Cited by (34)
Insights from the IronTract challenge: Optimal methods for mapping brain pathways from multi-shell diffusion MRI
2022, NeuroImageCitation Excerpt :These included regions where fibers branched into smaller bundles, or where they took a sharp turn to enter a bundle (Figs. 7 and 8, Supplementary video). These results agree with previous validation studies (Grisot et al., 2021; Schilling et al., 2019a) and illustrate the importance of anatomic tracing for identifying realistic failure modes of tractography that go beyond the simple crossing fiber configurations used in digital or physical phantoms. Almost all submissions were successful in identifying projections that ran through major crossing regions (Figs. 3 and 7).
Post mortem mapping of connectional anatomy for the validation of diffusion MRI
2022, NeuroImageCitation Excerpt :An early study that performed anatomic tracing and dMRI scanning in the same NHP brain was able to achieve a Dice coefficient (i.e., overlap) between tracing and tractography as high as 0.75, despite being limited by low spatial and angular resolution (Dauguet et al., 2007). Another dataset that included anatomic tracing and dMRI in the same animals was used to compare a wide range of tractography algorithms, including as part of the VoTEM challenge (Schilling and Gao,2019a; Schilling and Nath 2019b). These data confirm a result from (Thomas et al., 2014), that the default thresholds for probabilistic tractography operate in a regime with higher true and false positives than the default thresholds for deterministic tractography.
Mapping the human corticoreticular pathway with multimodal delineation of the gigantocellular reticular nucleus and high-resolution diffusion tractography
2022, Journal of the Neurological SciencesTractography dissection variability: What happens when 42 groups dissect 14 white matter bundles on the same dataset?
2021, NeuroImageCitation Excerpt :In order to ultimately harmonize the anatomical definition of tracts and standardize the bundle segmentation process, we propose a first step is to quantify this variability, and understand the similarities and differences in bundle segmentation results across protocols. There have been many works that benchmark or validate the anatomical accuracy of tractography, typically comparing against simulated data (Daducci et al., 2014, Neher et al., 2015, Maier-Hein et al., 2017), physical phantoms (Guevara et al., 2012, Perrin et al., 2005), animal tracer studies (Schilling et al., 2019, Donahue et al., 2016, Girard et al., 2020, Grisot et al., 2021, Schmahmann and Pandya, 2006), or cadaveric dissections (Forkel et al., 2014, Lawes et al., 2008, Sarubbo et al., 2013, Maffei et al., 2018, Hau et al., 2017). These have led to insight into the challenges and limitations of tractography, including the presence of false positive and false negative pathways and subsequent sensitivity/specificity tradeoff in accuracy (Maier-Hein et al., 2017, Schilling et al., 2019, Thomas et al., 2014, Aydogan et al., 2018, Knösche et al., 2015), and the presence of biases (Rheault et al., 2020) due to pathway shape and location (Yeh et al., 2016), anatomy (Schilling et al., 2018, Reveley et al., 2015), and processing decisions (Girard et al., 2014).