A full 360 $^\circ$ × 180 $^\circ$ image provides an unlimited field of view (FOV) and an immersive experience for the users without any loss of information of the surrounding. In this study, a deep learning based approach is proposed to synthesize a 360 $^\circ$ image from a sparse set of images captured with a limited FOV. The proposed network consists of a cascade of the FOV estimation network and

A multistage algorithm for quantitative microwave breast imaging is presented which utilizes the eigenfunction-based reconstruction of the complex-valued permittivity as prior information. The eigenfunction-based reconstruction is obtained from a single-frequency non-iterative microwave inversion technique that uses the eigenfunctions of the Helmholtz operator, in a resonant conductive enclosure, as

Mask-based lensless cameras replace the lens of a conventional camera with a custom mask. These cameras can potentially be very thin and even flexible. Recently, it has been demonstrated that such mask-based cameras can recover light intensity and depth information of a scene. Existing depth recovery algorithms either assume that the scene consists of a small number of depth planes or solve a sparse

In compressed sensing MRI (CS-MRI), k-space measurements are under-sampled to achieve accelerated scan times. CS-MRI presents two fundamental problems: (1) where to sample and (2) how to reconstruct an under-sampled scan. In this article, we tackle both problems simultaneously for the specific case of 2D Cartesian sampling, using a novel end-to-end learning framework that we call LOUPE (Learning-based

In recent years, photoacoustic image reconstruction has received extensive attention. Various reconstruction methods, such as back-projection, frequency domain reconstruction, time reversal, and model-based reconstructions have been developed. Although these methods are based on different propagation theories, they have relatively simple implementations when reconstructing images on homogenous media

At high frequencies, phase measurement is difficult to meet the accuracy required for imaging, and it is more susceptible to noise pollution. Consequently, it is important to develop inversion models and algorithms for electromagnetic inverse scattering problems with phaseless data (PD-ISPs). Compared to the ISPs with full-data (FD-ISPs), the PD-ISPs are more nonlinear due to the lack of the phase

Low-rank tensor completion recovers missing entries based on different tensor decompositions. Due to its outstanding performance in exploiting some higher-order data structure, low rank tensor ring has been applied in tensor completion. To further deal with its sensitivity to sparse component as it does in tensor principle component analysis, we propose robust tensor ring completion (RTRC), which separates

Deconvolution microscopy has been extensively used to improve the resolution of the wide-field fluorescent microscopy, but the performance of classical approaches critically depends on the accuracy of a model and optimization algorithms. Recently, the convolutional neural network (CNN) approaches have been studied as a fast and high performance alternative. Unfortunately, the CNN approaches usually

Recently, single gray/RGB image super-resolution reconstruction task has been extensively studied and made significant progress by leveraging the advanced machine learning techniques based on deep convolutional neural networks (DCNNs). However, there has been limited technical development focusing on single hyperspectral image super-resolution due to the high-dimensional and complex spectral patterns

Presents a listing of the editorial board, board of governors, current staff, committee members, and/or society editors for this issue of the publication.

Binary tomography is concerned with the recovery of binary images from a few of their projections (i.e., sums of the pixel values along various directions). To reconstruct an image from noisy projection data, one can pose it as a constrained least-squares problem. As the constraints are nonconvex, many approaches for solving it rely on either relaxing the constraints or heuristics. In this paper, we

In this paper, we present a full view optical flow estimation method for plenoptic imaging. Our method employs the structure delivered by the four-dimensional light field over multiple views making use of superpixels. These superpixels are four dimensional in nature and can be used to represent the objects in the scene as a set of slanted-planes in three-dimensional space so as to recover a piecewise

In micro-tomography, variants of helical X-ray source trajectories, e.g., double-helix (DH) and space-filling (SF), are attractive alternatives to the conventional circular trajectory, as they satisfy data-sufficiency conditions; this enables exact reconstruction, and large cone-angle (or high flux) imaging. Geometric alignment of micro-tomography experimental data, i.e., radiographs or projection

Removing the undesired reflections of images taken through glass is an important problem in digital photography and many other vision applications. The so-called ghosting effect, i.e., the pattern repetitiveness in reflection, is an effective cue used by existing techniques to remove reflection from images. Existing methods take a two-stage approach that first estimates the parameters of ghosting effect

In this paper, we introduce a deep-learning-based framework to solve electromagnetic inverse scattering problems. This framework builds on and extends the capabilities of existing physics-based inversion algorithms. These algorithms, such as the contrast source inversion, subspace-optimization method, and their variants face a problem of getting trapped in false local minima when recovering objects

Real-world data processing problems often involve various image modalities associated with a certain scene, including RGB images, infrared images, or multispectral images. The fact that different image modalities often share certain attributes, such as edges, textures, and other structure primitives, represents an opportunity to enhance various image processing tasks. This paper proposes a new approach

Coded aperture X-ray computed tomography (CAXCT) systems reconstruct high quality images of the inner structure of an object from a few coded illumination measurements. Since the computed tomography (CT) system matrix is highly structured, random coded apertures lead to lower quality image reconstructions. In this paper, the noisy forward models of CAXCT in both Gaussian noise and Poisson noise are

In this paper, we consider imaging the reflectivity of scatterers from intensity-only data recorded by a single moving transducer that both emits and receives signals, forming a synthetic aperture. By exploiting frequency illumination diversity, we obtain multiple intensity measurements at each location, from which we determine field cross correlations using an appropriate phase controlled illumination

When a narrowband coherent wavefront passes through or reflects off of a scattering medium, the input and output relationship of the incident field is linear and so can be described by a transmission matrix (TM). If the TM for a given scattering medium is known, one can computationally “invert” the scattering process and image through the medium. In this paper, we investigate the effect of broadband

In this paper, we consider the problem of vectorvalued regularization of light fields based on PDEs. We propose a regularization method operating in the four-dimensional (4-D) ray space that does not require prior estimation of disparity maps. The method performs a PDE-based anisotropic diffusion along directions defined by local structures in the 4-D ray space. We analyze light field regularization

Photon counting detection is a promising approach toward dose reduction in X-ray computed tomography (CT). Full CT reconstruction from a fraction of the detected photons required by energy-integrating detectors has been demonstrated. In medical and industrial CT applications, projection truncation to the region-of-interest (ROI) is another effective way of dose reduction, as information from the ROI

The challenges of real world applications of the laser detection and ranging (Lidar) three-dimensional (3-D) imaging require specialized algorithms. In this paper, a new reconstruction algorithm for single-photon 3-D Lidar images is presented that can deal with multiple tasks. For example, when the return signal contains multiple peaks due to imaging semitransparent surfaces, or when imaging through

Sparsity and low-rank models have been popular for reconstructing images and videos from limited or corrupted measurements. Dictionary or transform learning methods are useful in applications such as denoising, inpainting, and medical image reconstruction. In this paper, we propose a framework for online (or time-sequential) adaptive reconstruction of dynamic image sequences from linear (typically

This article demonstrates for the first time, both numerically and experimentally, the feasibility of radar-based microwave imaging of anthropomorphic heterogeneously dense breasts in prone position, requiring no immersion liquid. The dry, contactless approach greatly simplifies the setup, favors patient comfort, and further avoids lengthy sanitation procedures after each exam. We use a radar-type

Filtered back projection (FBP) is the most widely used method for image reconstruction in X-ray computed tomography (CT) scanners, and can produce excellent images in many cases. However, the presence of dense materials, such as metals, can strongly attenuate or even completely block X-rays, producing severe streaking artifacts in the FBP reconstruction. These metal artifacts can greatly limit subsequent

Multi-dimensional, multi-contrast magnetic resonance imaging (MRI) has become increasingly available for comprehensive and time-efficient evaluation of various pathologies, providing large amounts of data and offering new opportunities for improved image reconstructions. Recently, a cardiac phase-resolved myocardial T 1 mapping method has been introduced to provide dynamic information on tissue viability

Light detection and ranging (Lidar) single-photon devices capture range and intensity information from a three-dimensional (3-D) scene. This modality enables long range 3-D reconstruction with high range precision and low laser power. A multispectral single-photon Lidar system provides additional spectral diversity, allowing the discrimination of different materials. However, the main drawback of such

In through-the-wall radar imaging, wall clutter mitigation and image formation are often solved separately, resulting in a suboptimal solution. This paper presents a scene reconstruction model with low rank, sparsity, and total-variation constraints that simultaneously remove the wall clutter and form the image of the scene. The proposed method exploits the low rank property of the wall clutter to

Phase recovery from the bispectrum is a central problem in speckle interferometry which can be posed as an optimization problem minimizing a weighted nonlinear least-squares objective function. We look at two different formulations of the phase recovery problem from the literature, both of which can be minimized with respect to either the recovered phase or the recovered image. Previously, strategies

We propose a method for super-resolution (SR) of low-resolution (LR) color images taken in low-light scenes. Our method is based on the multi-frame SR technique, which reconstructs a high-resolution color image by fusing multiple LR images that were taken at different camera positions so as to alter the alignment between scene details and pixels. Previous multi-frame SR methods have implicitly assumed

Three-dimensional (3-D) radar imaging can provide additional information along elevation dimension about the target with respect to the conventional 2-D radar imaging, but usually requires a huge amount of data collected over 3-D frequency-azimuth-elevation space, which motivates us to perform 3-D imaging by using sparsely sampled data. Traditional compressive sensing (CS) based 3-D imaging methods

Good temporal representations are crucial for video understanding, and the state-of-the-art video recognition framework is based on two-stream networks. In such framework, besides the regular ConvNets responsible for RGB frame inputs, a second network is introduced to handle the temporal representation, usually the optical flow (OF). However, OF or other task-oriented flow is computationally costly

Recently, we developed a generalized phase gradient autofocus (GPGA) algorithm for performing synthetic aperture radar (SAR) autofocus over arbitrary flight paths, including both near-field and bistatic collection geometries. A key step of the GPGA algorithm is solving or finding an approximate solution to a non-deterministic polynomial-time hard (NP-hard) optimization problem, whose solution set consists

Light fields present a rich way to represent the 3D world by capturing the spatio-angular dimensions of the visual signal. However, the popular way of capturing light fields (LF) via a plenoptic camera presents a spatio-angular resolution trade-off. To address this issue, computational imaging techniques such as compressive light field and programmable coded aperture have been proposed, which reconstruct

Although various advanced super-resolution microscopies have been developed, conventional microscopes are still the most widely used. Therefore, it is still one of the goals of light microscopists to improve the imaging performance of conventional microscopes. Ernst Abbe had proven the resolution of a bright-field microscope can be improved by oblique illumination in 1873, but the approach has not

Many challenging image processing tasks can be described by an ill-posed linear inverse problem: deblurring, deconvolution, inpainting, compressed sensing, and superresolution all lie in this framework. Traditional inverse problem solvers minimize a cost function consisting of a data-fit term, which measures how well an image matches the observations, and a regularizer, which reflects prior knowledge

We study the problem of recovering structured data from Fourier ptychography measurements. Fourier ptychography is an image acquisition scheme that uses an array of images to produce high-resolution images in microscopy as well as long-distance imaging, to mitigate the effects of diffraction blurring. The number of measurements is typically much larger than the size of the signal (image or video) to

We study the design of a coding mask for pulse-echo ultrasound imaging. We are interested in the scenario of a single receiving transducer with an aberrating layer, or ‘mask,’ in front of the transducer's receive surface, with a separate co-located transmit transducer. The mask encodes spatial measurements into a single output signal, containing more information about a reflector's position than a

Endmember (EM) spectral variability can greatly impact the performance of standard hyperspectral image analysis algorithms. Extended parametric models have been successfully applied to account for the EM spectral variability. However, these models still lack the compromise between flexibility and lowdimensional representation that is necessary to properly explore the fact that spectral variability

Fluorescence laser-scanning microscopy is a well-established imaging technique in biology, available in many imaging facilities to investigate structures within live animal embryos such as zebrafish. Laser scanning microscopes (LSM) are limited when used to study dynamic heart morphology or function. Despite their ability to resolve static cardiac structures, the fast motion of the beating heart introduces

Frame-based optical flow (OF) methods struggle in the presence of large motion and occlusion due to slow frame rates. Optical flow of dynamic vision sensor (DVS) has gained attention recently as a way to overcome these shortcomings. DVS reports “events” corresponding log intensity changes exceeding a specific threshold, with an accuracy of microsecond order. The increased temporal sampling rate are

Near-infrared (NIR) band sensors capture digital images of scenes under special conditions such as haze, fog, overwhelming light or mist, where visible (VS) band sensors get occluded. However, the NIR images contain poor textures and colors of different objects in the scene, on the contrary to the VS images. In this article, we propose a simple yet effective fusion approach that combines both VS and

Full-waveform inversion problems are usually formulated as optimization problems, where the forward-wave propagation operator f maps the subsurface velocity structures to seismic signals. The existing computational methods for solving full-waveform inversion are not only computationally expensive, but also yields low-resolution results because of the ill-posedness and cycle skipping issues of full-waveform

To improve the compressive sensing MRI (CS-MRI) approaches in terms of fine structure loss under high acceleration factors, we have proposed an iterative feature refinement model (IFR-CS), equipped with fixed transforms, to restore the meaningful structures and details. Nevertheless, the proposed IFR-CS still has some limitations, such as the selection of hyper-parameters, a lengthy reconstruction

A novel cascade learning framework to incrementally train deeper and more accurate convolutional neural networks is introduced. The proposed cascade learning facilitates the training of deep efficient networks with plain convolutional neural network (CNN) architectures, as well as with residual network (ResNet) architectures. This is demonstrated on the problem of image super-resolution (SR). We show

Several recent studies on compressive video sensing realized scene capture beyond the fundamental trade-off limit between spatial resolution and temporal resolution using random space-time sampling. However, most of these studies obtained results for higher-frame-rate video that was produced in simulation experiments or using an optically simulated random sampling camera, because there are currently

Multi-energy computed tomography (MCT) has a great potential in material decomposition, tissue characterization, lesion detection, and other applications. However, the severe noise that exists within projections makes it difficult to obtain high-quality MCT images. To overcome this limitation, we propose a method termed Spectral-Image Similarity-based Tensor with Enhanced-sparsity Reconstruction (SISTER)

We analyze synthetic aperture radar (SAR) imaging of complex ground scenes that contain both stationary and moving targets. In the usual SAR acquisition scheme, we consider ways to preprocess the data so as to separate the contributions of the moving targets from those due to stationary background reflectors. Both components of the data, that is, reflections from stationary and moving targets, are

Inverse problems play a central role for many classical computer vision and image processing tasks. Many inverse problems are ill-posed, and hence require a prior to regularize the solution space. However, many of the existing priors, like total variation, are based on ad-hoc assumptions that have difficulties to represent the actual distribution of natural images. Thus, a key challenge in research

Positron emission tomography (PET) suffers from severe resolution limitations which reduce its quantitative accuracy. In this paper, we present a super-resolution (SR) imaging technique for PET based on convolutional neural networks (CNNs). To facilitate the resolution recovery process, we incorporate high-resolution (HR) anatomical information based on magnetic resonance (MR) imaging. We introduce

The light transport captures a scene's visual complexity. Acquiring light transport for dynamic scenes is difficult, since any change in viewpoint, materials, illumination or geometry also varies the transport. One strategy to capture dynamic light transport is to use a fast “flying-dot” projector; i.e., where an impulse light-probe is quickly scanned across the scene. We have built a novel fast flying-dot

In the real application scenario, the time information when a moving target enters/leaves the radar coverage is often unknown, which would lead to severe performance loss for target refocusing, parameter estimation and imaging. We address the refocusing and motion parameters estimation problem for a moving target with unknown entry/departure time, involving range cell migration (RCM) and Doppler frequency

The recently developed single-element surface rotating radio-frequency coil (RRFC) for magnetic resonance imaging (MRI) is able to acquire signals from around the subject in the manner of a multi-coil array, while exhibiting simplified construction and no coil size restrictions. Herein, we present a novel image reconstruction algorithm for data acquired with the RRFC, which we label WARF (a weighted-sum

There has been a growing attention to using acoustic-resolution photoacoustic microscopy (ARPAM) for medical applications in the recent decade. To eliminate the distortion induced by acoustic diffraction in ARPAM, synthetic aperture focusing technique (SAFT) methods that incorporate the virtual detector (VD) concept are used to reconstruct the images. However, most of these algorithms assume homogeneous

This article introduces and evaluates numerically a multigrid solver for non-linear tomographic radar imaging. Our goal is to enable the fast and robust inversion of sparse time-domain data with a mathematical full-wave approach utilizing a higher-order Born approximation (BA). Full-wave inversion is computationally expensive, hence techniques to speed up the numerical procedures are needed. To model

We quantitatively investigate multiple algorithms for microlens array grid estimation for microlens array-based light field cameras. Explicitly taking into account natural and mechanical vignetting effects, we propose a new method for microlens array grid estimation that outperforms the ones previously discussed in the literature. To quantify the performance of the algorithms, we propose an evaluation

Event cameras capture brightness changes at an ultra-high speed and output a series of asynchronous events. In order to form image frames without motion blur from event stream, motion compensation needs to be applied. Unlike other algorithms which either estimate ego-motion or estimate optical flow with strict assumptions, we directly align the curved event trajectories with time-varying motion parameters

Magnetic resonance imaging (MRI) is a widely used medical imaging modality. However, due to the limitations in hardware, scan time, and throughput, it is often clinically challenging to obtain high-quality MR images. The super-resolution approach is potentially promising to improve MR image quality without any hardware upgrade. In this article, we propose an ensemble learning and deep learning framework

An X-ray Talbot-Lau grating interferometer enables imaging of X-ray phase contrast. It can measure material transitions that are difficult to observe with traditional attenuation X-ray, for example soft-tissue variations in medical diagnosis or weakly absorbing materials in optical inspection. Unfortunately, the field of view of a Talbot-Lau interferometer is limited to few centimeters due to manufacturing

Visible images provide abundant texture details and environmental information, while infrared images benefit from night-time visibility and suppression of highly dynamic regions; itis a meaningful task to fuse these two types of features from different sensors to generate an informative image. In this article, we propose an unsupervised end-to-end learning framework for infrared and visible image fusion