We present a method for estimating, in real time, a 3D displacement field from a single fluoroscopic image. Our approach uses a fully convolutional network architecture to solve the associated inverse problem. Supervised learning is performed on synthetic data, using Digitally Reconstructed Radiographs as input and displacement fields as output. We use randomized Gaussian kernels to produce a synthetic

Federated learning (FL) has emerged as a transformative framework for privacy-preserving distributed training, allowing clients to collaboratively train a global model without sharing their local data. This is especially crucial in sensitive fields like healthcare, where protecting patient data is paramount. However, privacy leakage remains a critical challenge, as the communication of model updates

Graph neural networks (GNNs) are at the forefront of learning-based functional connectome analyses and neuropsychiatric disorder diagnoses with fMRI data. The reliability of the deployed GNNs largely depends on their explainability and generalizability, due to the complexity of functional brain networks and heterogeneous fMRI data acquisitions. While developing explainable or generalizable GNNs has

We propose an Advanced Surface Imaging (A-SI) framework to enable ultra-fast tumor localization for real-time tumor motion tracking in surface-guided radiotherapy (SGRT). SGRT uses optical surface imaging (OSI) to acquire ultra-fast surface topography images of the patient on the treatment couch. This serves as a surrogate for intra-fractional tumor motion tracking to guide radiation delivery. However

Accurate prediction of postoperative vault, the distance between the implantable collamer lens (ICL) posterior surface and the crystalline lens anterior surface, is critical for the success of ICL surgery. Existing regression-based prediction methods fail to provide visual postoperative observations, which are essential for a comprehensive risk assessment. Anterior segment optical coherence tomography

Contrast-enhanced knowledge distillation promises to transform medical diagnostics and reveal promising approaches for tumor segmentation on non-contrast medical images. However, existing methods related to contrast-enhanced knowledge distillation still make it hard to distill reliable contrast-enhanced knowledge for tumor segmentation due to the limitations of (1) unable to quantify uncertainty information

Simulating prospective magnetic resonance imaging (MRI) scans from a given individual brain image is challenging, as it requires accounting for canonical changes in aging and/or disease progression while also considering the individual brain’s current status and unique characteristics. While current deep generative models can produce high-resolution anatomically accurate templates for population-wide

Computer-aided surgical simulation is a critical component of orthognathic surgical planning, where accurately simulating face-bone shape transformations is significant. The traditional biomechanical simulation methods are limited by their computational time consumption levels, labor-intensive data processing strategies and low accuracy. Recently, deep learning-based simulation methods have been proposed

Few-shot semantic segmentation (FSS) methods have shown great promise in handling data-scarce scenarios, particularly in medical image segmentation tasks. However, most existing FSS architectures lack sufficient interpretability and fail to fully incorporate the underlying physical structures of semantic regions. To address these issues, in this paper, we propose a novel deep unfolding network, called

Cell segmentation, which involves separating individual cells in biomedical images, is essential for disease analysis and drug development research. However, many existing methods are restricted to specific types of images or require constant adjustment, making them time-consuming and labor-intensive. We introduce a new framework called Mimic, which employs a ”Query-and-Answer” (Q&A) mechanism to segment

Accurate instrument pose estimation is a crucial step towards the future of robotic surgery, enabling applications such as autonomous surgical task execution. Vision-based methods for surgical instrument pose estimation provide a practical approach to tool tracking, but they often require markers to be attached to the instruments. Recently, more research has focused on the development of markerless

Splenomegaly, the enlargement of the spleen, is an important clinical indicator for various associated medical conditions, such as sickle cell disease (SCD). Spleen length measured from 2D ultrasound is the most widely used metric for characterising spleen size. However, it is still considered a surrogate measure, and spleen volume remains the gold standard for assessing spleen size. Accurate spleen

Fetal growth restriction, affecting up to 10% of pregnancies, is a critical factor contributing to perinatal mortality and morbidity. Ultrasound measurements of the fetal abdominal circumference (AC) are a key aspect of monitoring fetal growth. However, the routine practice of biometric obstetric ultrasounds is limited in low-resource settings due to the high cost of sonography equipment and the scarcity

Contrastive pretraining can substantially increase model generalisation and downstream performance. However, the quality of the learned representations is highly dependent on the data augmentation strategy applied to generate positive pairs. Positive contrastive pairs should preserve semantic meaning while discarding unwanted variations related to the data acquisition domain. Traditional contrastive

Assessing risk factors for intracranial aneurysm (IA) development on images is crucial for early detection of high-risk cases. IAs often form at bifurcations within the circle of Willis (CoW), but manual assessment of these arteries is both time-consuming and susceptible to inconsistencies. Previous studies on imaging markers for IA development lack sufficient evidence for clinical implications, highlighting

Accurate and rapid Magnetic Resonance Imaging (MRI) is critical for clinical diagnosis. However, different sampling strategies and datasets act as confounding factors, significantly impacting the quality of image reconstruction. While existing methods can capture correlations between data during the imaging process, they overlook the deeper associations rooted in causal relationships. To address this

Recent advances in AI foundation models have significant potential for lightening the clinical workload by mimicking the comprehensive and multi-faceted approaches used by medical professionals. In the field of radiation oncology, the integration of multiple modalities holds great importance, so the opportunity of foundational model is abundant. Inspired by this, here we present RO-LMM, a multi-purpose

Artificial intelligence and deep learning are powerful tools for extracting knowledge from large datasets, particularly in healthcare. However, their black-box nature raises interpretability concerns, especially in high-stakes applications. Existing eXplainable Artificial Intelligence methods often focus solely on visualization or rule-based explanations, limiting interpretability’s depth and clarity

Recent advancements in surgical computer vision applications have been driven by vision-only models, which do not explicitly integrate the rich semantics of language into their design. These methods rely on manually annotated surgical videos to predict a fixed set of object categories, limiting their generalizability to unseen surgical procedures and downstream tasks. In this work, we put forward the

Multi-instance learning (MIL) exhibits advanced and surpassed capabilities in understanding and recognizing complex patterns within gigapixel histopathological images. However, current MIL methods for the analysis of the histopathological images still give rise to two main concerns. On one hand, vanilla MIL methods intuitively focus on identifying salient instances (easy-to-classify instances) without

Digital twins for cardiac electrophysiology are an enabling technology for precision cardiology. Current forward models are advanced enough to simulate the cardiac electric activity under different pathophysiological conditions and accurately replicate clinical signals like torso electrocardiograms (ECGs). In this work, we address the challenge of matching subject-specific QRS complexes using anatomically

Magnetic Resonance Elastography (MRE) quantifies soft tissue stiffness by measuring induced shear waves. MRE inversion techniques for parameter reconstruction are often affected by noise and compression waves. Neural network-based inversions have emerged as a possible solution to address these challenges. However, current approaches lack generalizability and do not provide uncertainty estimates. Therefore

Understanding the intricate patterns and interactions within functional brain networks (FBNs) is crucial for the accurate diagnosis and analysis of mental disorders. Brain function can be represented through different brain networks, each providing complementary insights into underlying neural processes. Integrating data from these different sources enables a more comprehensive and precise understanding

Recent advancements in machine learning (ML) allow for rapid analysis of complex image data, which supports the use of ultrasound (US)-based solutions in interventional procedures. These solutions often require large, labeled datasets that can be time-consuming to curate and subject to inter- and intra-labeler variability. This work presents a practical method for automated labeling of US images by

Recent technological advances in healthcare have led to unprecedented growth in patient data quantity and diversity. While artificial intelligence (AI) models have shown promising results in analyzing individual data modalities, there is increasing recognition that models integrating multiple complementary data sources, so-called multimodal AI, could enhance clinical decision-making. This scoping review

Multiple sclerosis (MS) is a chronic inflammatory disease characterized by demyelinating lesions in the central nervous system. Cross-sectional measurements of acute inflammatory lesion activity are typically obtained by detecting the presence of gadolinium enhancement in lesions, which typically lasts 3-6 weeks. We formulate the novel and clinically relevant task of quantification of recent acute

Dataset bias in images is an important yet less explored topic in medical images. Deep learning could be prone to learning spurious correlation raised by dataset bias, resulting in inaccurate, unreliable, and unfair models, which impedes its adoption in real-world clinical applications. Despite its significance, there is a dearth of research in the medical image classification domain to address dataset

Ultrasound imaging is pivotal in clinical diagnostics due to its affordability, portability, safety, real-time capability, and non-invasive nature. It is widely utilized for examining various organs, such as the breast, thyroid, ovary, cardiac, and more. However, the manual interpretation and annotation of ultrasound images are time-consuming and prone to variability among physicians. While single-task

Few-shot medical image segmentation typically uses a joint model for registration and segmentation. The registration model aligns a labeled atlas with unlabeled images to form initial masks, which are then refined by the segmentation model. However, inevitable spatial misalignments during registration can lead to inaccuracies and diminished segmentation quality. To address this, we developed EFS-MedSeg

The fetal standard plane acquisition task aims to detect an Ultrasound (US) image characterized by specified anatomical landmarks and appearance for assessing fetal growth. However, in practice, due to variability in human operator skill and possible fetal motion, it can be challenging for a human operator to acquire a satisfactory standard plane. To support a human operator with this task, this paper

Cancer survival prediction based on multimodal data (e.g., pathological slides, clinical records, and genomic profiles) has become increasingly prevalent in recent years. A key challenge of this task is obtaining an effective survival-specific global representation from patient data with highly complicated correlations. Furthermore, the absence of certain modalities is a common issue in clinical practice

Tertiary lymphoid structures (TLS) are ectopic lymphoid aggregates that form under specific pathological conditions, such as chronic inflammation and malignancies. Their presence within the tumor microenvironment (TME) is strongly correlated with patient prognosis and response to immunotherapy, making TLS detection in whole-slide pathological images (WSIs) crucial for clinical decision-making. Although

Invasive breast cancer (IBC) is a prevalent malignant tumor in women, and precise grading plays a pivotal role in ensuring effective treatment and enhancing survival rates. However, accurately grading IBC presents a significant challenge due to its heterogeneous nature and the need to harness the complementary information from multiple nuclei sources in histopathology images. To tackle this critical

Magnetic Resonance Imaging (MRI) produces images with different contrasts, providing complementary information for clinical diagnoses and research. However, acquiring a complete set of MRI sequences can be challenging due to limitations such as lengthy scan time, motion artifacts, hardware constraints, and patient-related factors. To address this issue, we propose a novel method to learn Contrast and

Super-resolution ultrasound (SRUS) imaging through localisation and tracking of microbubble (MB), also known as ultrasound localisation microscopy (ULM), allows non-invasive imaging of microvasculature in vivo beyond the diffraction limit. The number of MBs localised from the acquired contrast-enhanced ultrasound (CEUS) images and the localisation accuracy precision directly influence the quality of

Cardiovascular magnetic resonance imaging is emerging as a crucial tool to examine cardiac morphology and function. Essential to this endeavour are anatomical 3D surface and volumetric meshes derived from CMR images, which facilitate computational anatomy studies, biomarker discovery, and in-silico simulations. Traditional approaches typically follow complex multi-step pipelines, first segmenting images

Difficulty in resisting gravity is a common leg motor impairment in stroke patients, significantly impacting daily life. Automated clinical-level quantification of motor-leg videos based on the National Institutes of Health Stroke Scale is crucial for consistent and timely stroke diagnosis and assessment. However, real-world applications are challenged by interference impacting motion representation

Multiple instance learning (MIL) is a powerful approach for whole-slide pathological image (WSI) analysis, particularly suited for processing gigapixel-resolution images with slide-level labels. Recent attention-based MIL architectures have significantly advanced weakly supervised WSI classification, facilitating both clinical diagnosis and localization of disease-positive regions. However, these methods

Automatic radiology report generation can alleviate the workload for physicians and minimize regional disparities in medical resources, therefore becoming an important topic in the medical image analysis field. It is a challenging task, as the computational model needs to mimic physicians to obtain information from multi-modal input data (i.e., medical images, clinical information, medical knowledge

Recently, large pre-trained models (LPM) have achieved great success, which provides rich feature representation for downstream tasks. Pre-training and then fine-tuning is an effective way to utilize LPM. However, the application of LPM in the medical domain is hindered by the presence of a large number of parameters and a limited amount of labeled data. In clinical practice, there exists a substantial

Lung infections are the leading cause of death among infectious diseases, and accurate segmentation of the infected lung area is crucial for effective treatment. Currently, segmentation methods that rely solely on imaging data have limited accuracy. Incorporating text information enriched with expert knowledge into the segmentation process has emerged as a novel approach. However, previous methods

Image synthesis from Magnetic Resonance (MR) to Computed Tomography (CT) can estimate the electron density of tissues, thereby facilitating Positron Emission Tomography (PET) attenuation correction in whole-body PET/MR imaging. Whole-body MR-to-CT synthesis faces several challenges including the spatial misalignment caused by tissue variety and respiratory movements, and the complex intensity mapping

State-of-the-art research of traditional computer vision is increasingly leveraged in the surgical domain. A particular focus in computer-assisted surgery is to replace marker-based tracking systems for instrument localization with pure image-based 6DoF pose estimation using deep-learning methods. However, state-of-the-art single-view pose estimation methods do not yet meet the accuracy required for

Late gadolinium enhancement (LGE) cardiac magnetic resonance (CMR) imaging is considered the in vivo reference standard for assessing infarct size (IS) and microvascular obstruction (MVO) in ST-elevation myocardial infarction (STEMI) patients. However, the exact quantification of those markers of myocardial infarct severity remains challenging and very time-consuming. As LGE distribution patterns can

Accurate and efficient group-wise registration for medical images is fundamentally important to construct a common template image for population-level analysis. However, current group-wise registration faces the challenges posed by the algorithm’s efficiency and capacity, and adaptability to large variations in the subject populations. This paper addresses these challenges with a novel Nested Hierarchical

Endoscopic Submucosal Dissection (ESD) constitutes a firmly well-established technique within endoscopic resection for the elimination of epithelial lesions. Dissection trajectory prediction in ESD videos has the potential to strengthen surgical skills training and simplify surgical skills training. However, this approach has been seldom explored in previous research. While imitation learning has proven

Quality control (QC) of structures segmentation in volumetric medical images is important for identifying segmentation errors in clinical practice and for facilitating model development by enhancing network performance in semi-supervised and active learning scenarios. This paper introduces SegQC, a novel framework for segmentation quality estimation and segmentation error detection. SegQC computes

Medical segmentation models are evaluated empirically. As such an evaluation is based on a limited set of example images, it is unavoidably noisy. Beyond a mean performance measure, reporting confidence intervals is thus crucial. However, this is rarely done in medical image segmentation. The width of the confidence interval depends on the test set size and on the spread of the performance measure

Confounding factors inherent in medical images can significantly impact the causal exploration capabilities of deep learning models, resulting in compromised accuracy and diminished generalization performance. In this paper, we present an innovative methodology named CausalMixNet that employs query-mixed intra-attention and key&value-mixed inter-attention to probe causal relationships between input

In the domain of medical imaging, many supervised learning based methods for segmentation face several challenges such as high variability in annotations from multiple experts, paucity of labelled data and class imbalanced datasets. These issues may result in segmentations that lack the requisite precision for clinical analysis and can be misleadingly overconfident without associated uncertainty quantification

The absence of MRI sequences is a common occurrence in clinical practice, posing a significant challenge for prediction modeling of non-invasive diagnosis of glioma (GM) via fusion of multi-sequence MRI. To address this issue, we propose a novel unified reciprocal assistance imputation-representation learning framework (namely REPAIR) for GM diagnosis modeling with incomplete MRI sequences. REPAIR

Medical image segmentation is a pivotal task within the realms of medical image analysis and computer vision. While current methods have shown promise in accurately segmenting major regions of interest, the precise segmentation of boundary areas remains challenging. In this study, we propose a novel network architecture named CTO, which combines Convolutional Neural Networks (CNNs), Vision Transformer

This work presents a framework for monocular 6D pose estimation of surgical instruments in open surgery, addressing challenges such as object articulations, specularity, occlusions, and synthetic-to-real domain adaptation. The proposed approach consists of three main components: (1) synthetic data generation pipeline that incorporates 3D scanning of surgical tools with articulation rigging and physically-based

Virtual Imaging Trials (VIT) offer a cost-effective and scalable approach for evaluating medical imaging technologies. Computational phantoms, which mimic real patient anatomy and physiology, play a central role in VITs. However, the current libraries of computational phantoms face limitations, particularly in terms of sample size and heterogeneity. Insufficient representation of the population hampers

The recent popularity of foundation models and the pre-train-and-adapt paradigm, where a large-scale model is transferred to downstream tasks, is gaining attention for volumetric medical image segmentation. However, current transfer learning strategies devoted to full fine-tuning for transfer learning may require significant resources and yield sub-optimal results when the labeled data of the target