An in-depth analysis of hyperspectral target detection with shadow compensation via LiDAR

Highlights

• An in-depth analysis of hyperspectral target detection on targets in shadow.

• Use of LiDAR and HSI data in the physical radiance model for signature correction.

• Considerable improvement in shadow compensation via physics-based machine learning.

Abstract

Shadowy areas present a big hindrance in target detection from HSI, as the reflectance data received from target materials is significantly diminished when measured from shadowy areas. In this work, we perform an in-depth analysis of hyperspectral target detection on targets in full illumination and in partial or full shadows; and analyze how much target detection can be improved if the hyperspectral data is corrected at the regions of shadows. To do this, first, we detect the shadows using LiDAR, and propose a way to correct them in the hyperspectral image using the physical radiance model. Then, using three target detectors, namely the spectral angle mapper (SAM), adaptive coherence estimator (ACE) and matched filter (MF), we compare the results of target detection with and without shadow correction. We analyze our results based on the target material (red felt and blue felt targets), the background (grass or gravel), the amount of shadow (partial or full) and based on the time of the data collection (in the morning or at noon). Our results indicate several interesting observations: (i) the red-felt material is much harder to detect than the blue-felt material even though they are made up of the same material; but this gap in detection decreases significantly if shadow correction is performed using the radiance model, (ii) both the red-felt and the blue-felt targets are hard to detect earlier in the day when the rays from the sun are inclined; but there is not a significant difference in making the data collection in the morning or at noon if shadow correction is performed, and (iii) the shadow compensation dramatically increases the detection rates and boosts up the area under the receiver operating curve (AUC) from around 0,7-0,9 band to the 0,95-1,00 band.

In addition, we provide our shadow detection code¹, sky-view factor results and all the MODTRAN outputs for the parts of the Share2012 dataset used in this work. In doing so, we hope to provide a benchmark for researchers who would like to test their target detection or shadow correction algorithms on HSI-LiDAR data.

Introduction

Target detection lies at the very core of hyperspectral imaging (HSI) research. However, shadowy areas present a big hindrance in target detection from HSI, as the reflectance data received from target materials in the near infrared (NIR) and short wave infrared (SWIR) bands is significantly diminished when measured from shadowy areas [1], [2]. Since hyperspectral sensors use the sun as a source of radiation, the same material in a hyperspectral dataset can show a spectral variation based on the lighting conditions, which lead to decreased target detection rates in shadows. With the use of LiDAR, shadow finding algorithms based solely on HSI [3], [4], [5] were complemented with more accurate information [6], and the researchers gained the ability to use radiance models [7], [8], [9], [10]. The studies in [11], [12] make a detailed comparison of shadow pixels detected from HSI and from LiDAR. Apparent from these studies is the critical importance of the position of the sun in data collection, the effect of shadows on target detection and the advantages of using LiDAR.

In this work, our goal is to get a clear understanding of the effects of targets and illumination on detection. The first step in dealing with shadows is to locate the shadow regions, which can be efficiently done using LiDAR data [13], [14]. Therefore, hyperspectral and LiDAR data, complete with a set of atmospheric measurements and the information on the angles and altitudes of data collection, present a very unique medium for HSI research. Unfortunately, such data with good ground truths is very limited, and even if it exists, researchers have to first deal with shadow extraction from LiDAR. Further, if one wants to incorporate this to the hyperspectral radiance model, atmospheric modeling tools such as MODTRAN [15], [16] take ample time, and may not be available to everyone.

In this paper, we use the Share2012 dataset [17], deal with all of the issues listed above for this dataset, and share all the necessary codes and outputs on the web to facilitate rapid and comparable research. Further, we propose to perform shadow correction on hyperspectral data using LiDAR and explain it on an elementary guide to the radiance model. Then, we show that shadow correction using LiDAR data significantly increases HSI target detection rates. For this purpose, we provide the results of the spectral angle mapper (SAM), adaptive coherence estimator (ACE) and matched filter (MF) detectors and compare their outputs on both the reflectance dataset of Share2012 (provided with the dataset based on atmospheric correction with ATCOR), and on our reflectance dataset which is obtained through shadow and atmospheric correction. Finally, we investigate if data collection in the morning or in the evening makes a difference; and how much the color or the background of the target affect the results. With these results, we provide a benchmark for HSI-LiDAR exploration.

The rest of the paper is organized as follows: we present the related work in Section 2, and discuss the details of the proposed method in Section 3 with an elementary guide to the radiance model in Section 3.1. We explain the dataset and the targets detection methods used to evaluate our results in Section 4. Then we give the experimental results in Section 5 and conclude in Section 6.