Small Target Radiometric Performance of Drone-Based Hyperspectral Imaging Systems

Abstract

Hyperspectral imaging systems frequently rely on spectral rather than spatial resolving power for identifying objects within a scene. A hyperspectral imaging system’s response to point targets under flight conditions provides a novel technique for extracting system-level radiometric performance that is comparable to spatially unresolved objects.The system-level analysis not only provides a method for verifying radiometric calibration during flight but also allows for the exploration of the impacts on small target radiometry, post orthorectification. Standard Lambertian panels do not provide similar insight due to the insensitivity of orthorectification over a uniform area. In this paper, we utilize a fixed mounted hyperspectral imaging system (radiometrically calibrated) to assess eight individual point targets over 18 drone flight overpasses. Of the 144 total observations, only 18.1% or 26 instances are estimated to be within the uncertainty of the predicted entrance aperture-reaching radiance signal. For completeness, the repeatability of Lambertian and point targets are compared over the 18 overpasses, where the effects of orthorectification drastically impact the radiometric estimate of point targets. The unique characteristic that point targets offer, being both a known spatial and radiometric source, is that they are the only field-deployable method for understanding the small target radiometric performance of drone-based hyperspectral imaging systems.

1. Introduction

For drone-based imaging platforms, radiometric calibration is accomplished using Lambertian targets (i.e., Empirical Line Method (ELM) [1,2]) or a physics-based approach (i.e., At-Attitude Radiance Ratio (AARR) [3,4]). Both techniques have the goal of compensating for and calibrating remote sensing imagery to surface reflectance. The transformation of remote sensing imagery to surface reflectance more readily aids in the identification of materials.

During typical field data collection campaigns, the relative reflectance of Lambertian calibration targets are measured with respect to reference panels of high reflectivity (e.g., 99% Spectralon^TM). The absolute radiometric information of these panels, albeit being calibration targets or reference panels, requires additional effort and complexity, however. The complexities reside in the understanding of the true reflectance of any Lambertian panel (in the field) in the presence of directional and hemispherical sources of illumination, for example [5]. That said, an advantage of field-deployed Lambertian targets is their large spatial extent, where statistical averaging can be employed to improve the signal-to-noise (SNR) of calibration curves for reflectance transformation. Even though high SNR can be achieved over Lambertian targets, these calibration curves are blindly applied to spatially unresolved targets and often not Lambertian [6]. Assessing the small target radiometric performance from Lambertian calibration targets cannot be achieved due to their needed size and because many sub-pixel limitations are hidden within their high uniformity.

A physics-based approach to reflectance calibration uses remote sensing principles to transform imagery from entrance aperture-reaching spectral radiance to surface reflectance without in-scene calibration targets. The AARR technique uses the fundamental assumption that all objects in the scene are Lambertian reflectors such that the instantaneous measurements of the downwelling spectral irradiance (with a spectroradiometer) and object spectral radiance (with HSI system) can be used to estimate surface reflectance [4]. An advantage of this technique is the ability to correct hyperspectral (HS) imagery without in-scene calibration targets. However, this heavily relies on the calibratability of both instruments and the ability to accurately match different instruments (e.g., spectroradiometer to an HSI system). This calibration approach is limited to surface reflectance transformation, and no research has demonstrated the ability to examine HS radiometric performance.

Key design trade-offs for HSI systems prioritize radiometric sensitivity and spectral resolution over spatial acuity, resulting in spatially aliased imagery [6]. This puts more reliance on the spectral identification of spatially unresolved objects within the scene. The application space that challenges HSI systems is within the detection of sub-pixel or small targets [7]. Within this application space, small targets cannot be averaged over multiple pixels like an extended target (e.g., Lambertian calibration target). Furthermore, extracting small target radiometric performance from Lambertian targets does not provide adequate analysis in comparison to the application space [8]. Thus, an emphasis now exists on the radiometric performance of HSI systems in the presence of sub-pixel or small targets. The deployment of point targets allows users the ability to radiometrically examine collected imagery for troubleshooting both their instrument and post-processing techniques (e.g., orthorectification process). From what we have seen, no other vicarious technique can support the analysis and understanding of HS imagery for applications involving small targets. Furthermore, there is a lack of vicarious techniques for assessing the radiometric accuracy and performance of orthorectified products at the pixel level for drone-based HSI systems.

The only vicarious method that can investigate the radiometric performance of HSI systems to radiometrically known small targets is the SPecular Array for Radiometric Calibration (SPARC) technique [9]. The SPARC technique produces a radiometrically accurate point target for assessing the small target radiometric performance of imaging systems. More importantly, when assessing the small target radiometric performance of a drone-based HSI system, we note that orthorectification is required as a post-processing step for map** pixel location to a true ground position at a uniform pixel scale. Unfortunately, there are inaccuracies within this post-processing step from interpolation schemes (e.g., nearest neighbor) related to the assumption of a uniform pixel scale (i.e., that every pixel is square) [10]. Furthermore, inaccuracies accumulate from platform motion, HS mounting schemes, boresight offsets, sensor models, resolution scales within Digital Elevation Models (DEMs), and the absolute accuracy of Global Positioning Systems (GPSs)/Inertial Measurement Units (IMUs). All these inaccuracies manifest into an aggregate system level problem that requires evaluation utilizing field experiments and cannot be addressed separately due to the unforeseen correlations. Testing, characterizing, and understanding these system level effects using the SPARC method, as it relates to small target radiometric performance, is thus the subject of this paper.

The objective of this work is to demonstrate the extraction of small target radiometric performance for drone-based HSI systems through the use of convex mirrors. The SPARC technique offers a unique advantage over Lambertian targets since convex mirrors provide a radiometrically known point target. The deployment of characterized convex mirrors with supporting field spectroradiometers tested the absolute radiometric response and repeatability of a drone-based HSI system to small targets. When compared to Lambertian targets, the radiometric repeatability varied significantly from image to image, even though the HS instrument was radiometrically accurate. In general, the experiment demonstrated that the HS instrument tends to overestimate the radiometric signal from small targets due to errors introduced in the orthorectification process. The interpolation schemes used within orthorectification is theorized to account for many artifacts, which can change the spectral signature and introduce energy that was never imaged.

This paper is outlined as follows. Section 2 gives a background overview of our drone-based imaging platform and laboratory validation efforts of the HSI system. It then goes into detail about the theory behind the quantities that formulate the SPARC technique from a radiometric and spatial perspective. Section 3 speaks to the experiment overview, including the methodologies used in the field measurements and extraction of the radiometric signals related to Lambertian and point targets. Section 4 presents the results, where Lambertian and point targets are assessed over the entire experiment. In this section, the radiometric performance of small targets, after orthorectification, is scrutinized. Lastly, Appendix A and Appendix B outline a full derivation of the SPARC equation (i.e., entrance aperture-reaching spectral radiance) and the propagated uncertainty.

2. Background

2.1. Equipment Overview

The Digital Imagery and Remote Sensing (DIRS) Laboratory at the Rochester Institute of Technology uses a revolutionary multi-modal UAV payload for remote sensing applications. This includes the simultaneous observation of four different imaging modalities, which include Visible-Near Infrared (VNIR) hyperspectral and multispectral cameras, a Long-Wave Infrared (LWIR) camera and a Light Detection and Ranging (LIDAR) sensor (see Figure 1). This drone-based payload provides an indispensable resource for solving challenging remote sensing problems, including those which are the subject of this paper. Specifically, the hyperspectral drone mounted instrument under investigation was VNIR HSI systems. Raw imagery was collected and pre-processed (using the manufacturers standard processing software) to account for dark offsets, dark currents, and dark signal non-uniformity. Flat-fielding was performed after dark corrections. This includes photon response non-uniformity, irradiance fall-offs and signal non-linearity. The final pre-processing step includes a radiance correction (derived by the the manufacturer) that converts digital numbers (DNs) to spectral radiance. Lastly, all HS imagery was orthorectified and interpolated [11].

As is standard practice within the DIRS Laboratory, all radiometric instruments are internally verified with a NIST-traceable Labsphere Inc., North Sutton, NH, USA 20 inch integrating sphere. The typical routine for validating radiometrically calibrated instruments involves observing the NIST-traceable Quartz-Tungsten Halogen (QTH) lamp (∼±1% (k = 2) in the VNIR) at four different light levels [12]. The radiometric validation demonstrates that the HSI system under investigation in this paper is well calibrated to spectral radiance within the expected uncertainty of the integrating sphere (see Figure 2). The HSI system under test was radiometrically calibrated by the manufacturer, where the absolute uncertainty and measurement errors are unknown. However, the knowledge of our integrating sphere can inform the accuracy of the spectral radiance measured with the HSI system to be within ±5%. During all our outside experiments, we assumed that our HSI system holds this uncertainty and is consistent across the field-of-view (FOV).

Other field equipment (e.g., an ASD field spectroradiometer) was also used during our experiments for predicting the at-aperture reaching spectral radiance of point targets. The ASD was also cross-validated using our integrating sphere and holds an absolute uncertainty of approximately ±2% across the VNIR spectrum. The ASD instrument was used to measure the total downwelling spectral irradiance (utilizing an advanced cosine corrector) and provides atmospheric monitoring capabilities that ensure consistency of the point target predictions.

2.2. Field Irradiance Measurement Theory

Predicting the entrance aperture-reaching spectral radiance from point targets relies on both the total downwelling irradiance and the amount of diffuse sky contributing to the measured signal [5]. In this section, we define the field irradiance measurement followed by the formal spectral radiance description of point targets in Section 2.3.

The ASD field spectroradiometer uses an advanced cosine corrector (i.e., optical diffuser with a collecting solid angle of 2$\pi$ steradians) to measure the total downwelling irradiance $E_T\left(\lambda \right)$. This can be represented mathematically as

$$\begin{array}{c}\hfill E_T\left(\lambda \right)=E_{solar}\left(\lambda \right)+E_{sky}\left(\lambda \right)\end{array}$$

(1)

where $E_{solar}\left(\lambda \right)$ is the direct solar irradiance and $E_{sky}\left(\lambda \right)$ is the sky irradiance. We assume no additional scattering from surrounding objects such as vegetation or man-made objects. To account for the contribution from the sky irradiance into the point targets spectral radiance signal, the sky irradiance term must be estimated. Without a precise instrument to perform this task (i.e., rotating shadowband radiometer which performs shading), the sky irradiance can be estimated manually by blocking (i.e., shading) the solar disk with a black object attached to a pole [13]. Even though this provides a good estimate of the Global-to-Diffuse Ratio, a side-shaded measurement would improve the accuracy. Side shading corrects for the signal lost during the sun-blocked measurement. This method was not implemented; however, we did take into account conservative estimates of uncertainty [14].

With estimates of the total downwelling irradiance and the sky irradiance (i.e., the shaded measurement), the Global-to-Diffuse Ratio, $G\left(\lambda \right)$ can be defined as

$$\begin{array}{c}\hfill G\left(\lambda \right)=\frac{E_{sky}\left(\lambda \right)}{E_T\left(\lambda \right)}=\frac{E_{sky}\left(\lambda \right)}{E_{solar}\left(\lambda \right)+E_{sky}\left(\lambda \right)}\end{array}$$

(2)

Equation (2) is the fractional contribution of the sky irradiance to the total downwelling irradiance and includes the instantaneous scattering properties of the atmosphere [15]. The Global-to-Diffuse Ratio $G\left(\lambda \right)$ is calculated from field measurements and is used in predicting the entrance aperture-reaching spectral radiance from convex mirrors and will be discussed in the following section.

2.3. Imaging Point Targets: Radiometric and Spatial Response

Imaging point targets (i.e., SPARC) with remote sensing instruments has been in development since 2010 [8,9,15,16] with more recent engineering advances, by Labsphere Inc., to bring an automated calibration service to the satellite community [17,18,19,20,21]. Further advancements have been explored by the Rochester Institute of Technology on the application of point targets for drone-based HSI systems [22,23]. In this section, we will summarize the radiometric and spatial response of an imaging system to a point target. A full derivation related to the radiometric properties of convex mirrors is presented in Appendix A.

A point target is created by the virtual image of the solar disk created by a convex mirror. The convex mirror has intrinsic properties defined by the surface reflectance ${\rho }_m\left(\lambda \right)$, radius of curvature $R_m$, and clear aperture diameter $D_m$. For remote imaging systems, a convex mirror (i.e., point targets) gives us both a radiometric and spatial calibration target, which is discussed below.

An imaging system’s spatial response to point targets is defined by the Sampled Point Spread Function (SPSF). The total radiometric energy from a point target spreads over the SPSF. The radiometric integral or summation over the SPSF defines the ensquared energy. Because the SPSF can be measured with convex mirrors during field experiments, the SPSF estimate will include not only the optical system and detector contributions but also motion blur, jitter and post-processing artifacts from orthorectification. Post-processing artifacts will be defined as orthorectification radiometric errors and will contribute to the small target radiometric performance analysis. Theoretically, the SPSF has an infinite spatial extent but the imaging system’s Noise-Equivalent Irradiance (NEI) and point target signal intensity will ultimately limit the spatial extent of the SPSF. As long as the observed signal from a point target is bright enough, the energy lost to noise will be negligible.

When downwelling irradiance, $E_T\left(\lambda \right)$ illuminates a convex mirror, and a virtual solar image is formed at the focal length (i.e., $f=-R/2$). In addition to reflected solar irradiance, a considerable amount of sky irradiance (defined by $G\left(\lambda \right)$) is observed. A point target is radiometrically defined by its spectral radiant intensity $I_m\left(\lambda \right)$ reflecting off the surface of the mirror. The mirror’s spectral radiant intensity is propagated to the entrance aperture-reaching spectral radiance $L_{EAR}\left(\lambda \right)$ as

$$\begin{array}{c}\hfill L_{EAR}\left(\lambda \right)=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$4$}\right.{\rho }_m\left(\lambda \right)R_m^2\left[1-G\left(\lambda \right)cos\left(2{\theta }_m\right)\right]\frac{E_T\left(\lambda \right)}{GS{D}^2}\end{array}$$

(3)

where $G\left(\lambda \right)$ is the Global-to-Diffuse Ratio, 2${\theta }_m$ defines the angle between the virtual solar image and optical axis, and $GSD$ is the imaging system’s ground sampling distance. In theory, Equation (3) is a point target’s total energy when summed over the SPSF, and it will be connected to the measured data in Section 3.3. This equation has the following assumptions: the atmospheric transmission loss from the target to the sensor is assumed to be negligible for UAV altitudes [3], both the solar irradiance and $G\left(\lambda \right)$ are measured throughout the experiment, and the ground sampling distances are related to square pixels. In addition, path radiance is negated from a point target’s signal during the image extraction; this will be further discussed in Section 3.3. Note that if $cos\left(2{\theta }_m\right)$ goes to zero (i.e., the mirror reflects the entire hemisphere), there is no need to measure the sky irradiance or estimate $G\left(\lambda \right)$ because the ASD field spectroradiometer will be measuring the same solid angle that the mirror reflects.

3. Methodology

3.1. Experiment Overview

In this section, we will estimate the spectral radiance of point targets (with a fixed mounted HSI system) so as to demonstrate the usability of convex mirrors to understand the energy conservation (or lack thereof) within orthorectified imagery.

Our experiment was designed to use industry standard Lambertian targets and point targets to investigate the radiometric properties of orthorectified HSI imagery. The Lambertian targets include three field-deployable Permaflect panels, manufactured by Labsphere Inc., with average hemispherical reflectance factors of 7% (dark gray), 25% (gray), and 50% (white). The selection of Lambertian targets is based on previous studies indicating that the global VNIR reflectance levels of natural land surfaces are typically less than 60% [24]. The point targets were protective aluminum-coated convex mirrors with a 25 mm radius of curvature (−12.5 mm focal length) and average specular reflectance (i.e., spectrally flat) greater than 85% from Edmund Optics. All properties related to the mirrors were identical (i.e., radius of curvature and reflectance) and independently measured by Labsphere Inc. The point targets were selected such that saturation did not occur for any pixel, which can be achieved by having the GSD and radius of curvature close in value.

During collection, the drone made simple down and back overpasses with all targets deployed close to the center of the sensor’s FOV. The drone flew at a nominal height of 32 m, resulting in a nominal GSD of 2.1 cm. The integration time was optimized (i.e., spectral signal peaks around 75% saturation) to 5.5 ms from a nadir observation of a secondary 50% Permaflect panel prior to take-off. The overpasses began at 11:15 a.m. local time with all 18 overpasses completed within 8 min. The solar zenith at the time of collection was 42 degrees. All images used were orthorectified using the same DEM, instrument calibration data, and processing software.

Both the point target configuration and Lambertian target layout can be seen in Figure 3. The Lambertian panels were placed off to the side of the point targets. The larger dark gray and white panels were 1 m × 1 m in size, whereas the gray panel was 0.5 m × 1 m. With a GSD of 2.1 cm, there is a minimum of 23 full pixels on the panels. The point targets are identified by a mirror ID. For example, the label 12.5MM-2 denotes mirror number 2 with a focal length of 12.5 mm. The point targets were deployed on a low reflectance (roughly 2%) background panel (0.9 m × 1 m) in grou**s of four with a minimum of 14 pixels between mirrors. The ASD field spectroradiometer, not shown in this image, was placed nearby in the middle of a cleared field away from the experiment area.

3.2. Field Irradiance Measurements

The spectral radiance from a point target is dependent on two field measurements: the total downwelling irradiance (Figure 4a) and the sky irradiance (Figure 4b). The sky irradiance measurement will be used to estimate the Global-to-Diffuse Ratio as previously mentioned. Both field measurements contained a 2% uncertainty on the radiometric calibration and additional contributions from variations in illumination over the duration of the experiment. The illumination variation was estimated by assessing the deviation of both measurements during the 8 min experiment. This was a small contribution to the field measurement uncertainty compared to the assumed 2% calibration uncertainty. As can be seen in Figure 4, the black curves demonstrate a low percent variation in total downwelling irradiance and atmospheric conditions during the experiment. Any changes in atmospheric conditions, such as clouds or particulate scattering, would be captured in both measurements.

As mentioned in Section 2.2, the sky irradiance was measured by shading the ASD cosine corrector from the solar disk. The shadowing device was a square piece of wood covered in black felt (similar to the mirror background material).

Estimating the point targets entrance aperture-reaching spectral radiance is defined in Equation (3). The total downwelling irradiance can be used directly, whereas Equation (2) defines the Global-to-Diffuse Ratio. Figure 5 contains the two field measurements used to estimate the predicted point target spectral radiance (i.e., Equation (3)). Depending on the amount of diffuse sky reflected into the sensor’s FOV, the point targets will have a similar spectral shape to the total downwelling irradiance. The convex mirrors merely act as an attenuator of the reflected energy as a function of the mirror coating and mirror geometry. Since all mirrors are in close proximity, all point target signal estimates use the same total downwelling irradiance and Global-to-Diffuse Ratio. The only difference is in the modification of the Global-to-Diffuse Ratio based on the unique mirror geometries (i.e., ${\theta }_m$).

3.3. Data Processing and Analysis

For each mirror, we predict the entrance aperture-reaching spectral radiance, using Equation (3), which is then compared to the collected HS, radiance calibrated, imagery. The uncertainty of Equation (3) can be derived and evaluated by following the Guide to Uncertainty Measurement framework [25]. This requires taking the partial derivative of Equation (3) with respect to all variables assuming small variations about the mean and negligible non-linear behavior. When all the variables have an estimated absolute uncertainty, the combined uncertainty for the entrance aperture-reaching spectral radiance from a point target can be estimated. The derivation of uncertainties, related to Equation (3), can be found in Appendix B.

Table 1. Example of relative uncertainties for all input variables, left of vertical bar, along with the estimated relative combined uncertainty (i.e., predicted entrance aperture-reaching spectral radiance), right of vertical bar.

${\displaystyle \frac{\mathit{u}\left({\mathit{\rho }}_{\mathit{m}}\right)}{{\mathit{\rho }}_{\mathit{m}}}}$	${\displaystyle \frac{\mathit{u}\left({\mathit{R}}_{\mathit{m}}\right)}{{\mathit{R}}_{\mathit{m}}}}$	${\displaystyle \frac{\mathit{u}\left({\mathit{D}}_{\mathit{m}}\right)}{{\mathit{D}}_{\mathit{m}}}}$	${\displaystyle \frac{\mathit{u}\left(\mathit{G}\right)}{\mathit{G}}}$	${\displaystyle \frac{\mathit{u}\left({\mathit{E}}_{\mathit{T}}\right)}{{\mathit{E}}_{\mathit{T}}}}$	${\displaystyle \frac{\mathit{u}\left(\mathit{GSD}\right)}{\mathit{GSD}}}$	${\displaystyle \frac{{\mathit{u}}_{\mathit{c}}\left({\mathit{L}}_{\mathit{EAR}}\right)}{{\mathit{L}}_{\mathit{EAR}}}}$
3%	2%	2%	2.06%	2.05%	3%	8.01%

provides an example of the maximum relative uncertainty of all input variables as well as the predicted entrance aperture-reaching spectral radiance.

The predicted entrance aperture-reaching spectral radiance (Figure 6a) and its respective uncertainty (Figure 6b) are plotted for each mirror. Mirror-to-mirror variations are less than 4% across the VNIR spectrum. It was determined that the mirror reflectance and radius of curvature were the main contributors to this difference. In this investigation, when comparing two different mirrors, we should not observe differences in the radiometric signal greater than the mirror-to-mirror variation. The assumed 2% uncertainty in the radiometric calibration of the ASD field spectroradiometer results in the uncertainty resembling the point target signal. The relative combined uncertainty of the spectral radiance predictions is around 8% as seen in Table 1.

The Permaflect panels shown in Figure 7 demonstrate two key concepts about the experimental conditions, consistency and repeatability. The consistency and repeatability of the spectral radiance reflecting off the panels from the 18 overpasses demonstrate the stability of the atmosphere and the HS instrument. It is important that the stability of the atmosphere is well understood with measurements from the ASD field spectroradiometer and reflected signals off the Lambertian targets. The measurement variation observed in Figure 4 (i.e., black curve) and the tight signal variations reflected off the Lambertian targets over the 18 overpasses (Figure 7) demonstrate this stability. It is critical that the HS instrument and atmosphere are stable during this investigation so as to eliminate these factors that could potentially contribute to additional variation in the predicted entrance aperture-reaching spectral radiance of the point targets.

The mirror background signal is an additive contribution to the point targets signal and is subtracted during the point target extraction process. The mirror background spatial uniformity and reflectance contribute to the ability to accurately isolate the point target predicted spectral radiance. The low reflectance background (i.e., ∼2% average hemispherical reflectance) provides the best scenario for optimizing the extraction accuracy. It is critical to fully understand the background stability and spatial variation during the experiment, for this could lead to a signal bias in the point target. Figure 8 contains an average spectral radiance measurement of the two mirror backgrounds, “Bkg1” and “Bkg2”. For each overpass, the mirror background is extracted at the center of the panel and averaged over roughly a 5 × 5 pixel box. The black felt material used to construct the multi-layer mirror background provides for excellent spatial uniformity (i.e., has volumetric scattering properties).

The point target entrance aperture-reaching spectral radiance can be isolated from the backgrounds by defining a small bounding box around the point target that contains all of the measurable signal. Equation (4) defines the ensquared energy from a point target, over a bounding box with N pixels, after an average background signal is subtracted:

$$\begin{array}{c}\hfill L_{mirror}\left(\lambda \right)=\sum _{p=1}^N\left(L_{img}(p,\lambda )-\overline{L_{bkg}\left(\lambda \right)}\right)\end{array}$$

(4)

An overlooked feature within Equation (4) is the averaged background subtraction. This process is critical for extracting only the point target’s signal. We also note that the background subtraction process additionally eliminates the path radiance, including adjacency effects. Because the point target produces a spatial response of the imaging system, the ensquared energy is spread over the SPSF, and only the ensquared energy can be compared to the predicted entrance aperture-reaching spectral radiance as defined in Equation (3). Figure 9 illustrates the result of Equation (4), where a point target within an optimized bounding box highlighted in orange is summed after background subtraction. If the background signal is accurately estimated, a varying bounding box will not contribute to the ensquared energy but will only impact the noise within the signal (i.e., Bienaymé’s identity). Because small targets will be assessed in a similar way to the point targets, the ensquared energy defines the limit of radiometric target detection from the background.

4. Results

This section presents the main findings and results of the experiment. In our analysis below, standard orthorectification processes are assumed. However, the exact process and interpolation schemes are irrelevant to fully understand and quantify the results because point targets provide for a “black box” approach. In fact, instrument and post-processing anonymity further demonstrates the usefulness of the SPARC technique. Nothing more than the ground sampling distance (GSD) is required to characterize small target radiometric performance. Analyzing the unorthorectified imagery provides a baseline to determine how much energy was originally collected for comparison with the orthorectified results. More importantly, the observed spatial pattern of point targets helps to aid in the discussion of the observed results.

The experimental design is key for extracting two types of results discussed in this section, highlighted in Figure 10. When the small point target is imaged over multiple frames (e.g., over 18 overpasses), the radiometric repeatability of small targets can be examined, as seen in Figure 10a, whereas when two similar point targets are imaged together, as seen in Figure 10b over a small region, the radiometric stability from orthorectification can be assessed. Both assessment strategies will allow for the investigation of radiometric errors induced on point targets by platform motion and orthorectification. Over the entire experiment, eight point targets were imaged during 18 overpasses, accounting for 144 point target observations in total. Of the 144 observations, only 26 point targets (or 18.1%) were within the uncertainty of the predicted entrance aperture-reaching spectral radiance, at a 95% confidence level.

When assessing the use of a new technology (i.e., point targets) for investigating the radiometric accuracy of drone-based HSI systems, it is critical to compare results to standardized methods (i.e., Lambertian targets). In our experimental plan, Lambertian panels were deployed that spanned roughly 2% to 50% reflectance to compare image-to-image radiometric stability. Figure 11a illustrates the spectral radiance standard deviation of each mirror given 18 observations (i.e., overpasses). Figure 11b illustrates the spectral radiance standard deviation of a single pixel, for each of the four Lambertian panels, given 18 observations.

Figure 12 is an observation of a single mirror’s spectral radiance across the 18 overpasses. A general trend exists when observing the difference between the unorthorectified (top) and orthorectified (bottom) imagery: the absolute radiometric signal of small targets is generally overestimated for fixed mounted HSI systems.

Figure 13 takes a closer look at two instances of the same point target (mirror ID: 12.5MM-4) imaged within 2 min or three overpasses apart. The solid blue curve represents the point target imaged on the second overpass of the experiment and provides a mirror signal estimate within the uncertainty. However, not even 2 min later, the same mirror (now in solid orange curve) is estimated to be 25% brighter.

If we focus our attention to a mirror pair as represented in Figure 10b, an examination of oversampling the observed scene can be discussed further. Figure 14 demonstrates a major issue: imaging point targets with a fixed mounted pushbroom HSI system encountering significant platform motion. When point targets are re-imaged over multiple frames due to platform motion, the measured radiometric response will always be overestimated. The comparison between mirror pairs shows that orthorectification can be different over a small region on the ground.

The previous examples demonstrated the inconsistent radiometric performance of orthorectification on the overall signal level. In the following examples, spectral distortions from the orthorectification process will be examined.

Figure 15 illustrates a mirror pair after orthorectification, where we can see that the observed spectral radiance has changed. When a scene is not imaged properly (i.e., along-track is spatially undersampled), orthorectification can accurately represent the radiometric signal from a point target or inadvertently eliminate energy. Within Figure 15, the pixel highlighted with the red “X” is completely eliminated after orthorectification. The red arrows indicate full row interpolation after the imagery is orthorectified. Mirror 1 (blue curves) is a perfect example of linear interpolation positively affecting the mirror’s spectral radiance. Whereas, mirror 2 (orange curves) demonstrates a major issue with the orthorectification as energy was eliminated.

Figure 16 highlights issues when only single pixels are linearly interpolated and contributes to spectral inconsistencies in the spectral radiance of the orthorectified imagery. Single-pixel interpolation can increase the ensquared energy by forming linear combinations of surrounding pixels that do not spectrally match the target. The ensquared energy collected in the unorthorectified imagery was within the predicted uncertainty (black line), but the orthorectification process added new pixels (red boxes on right image within the figure) that spectrally changed the overall shape.

5. Discussion

In this section, we discuss three main aspects of our results. Firstly, we compare the standard deviation of all eight point targets to all Lambertian targets across all 18 overpasses. This provides the first glimpse into the radiometric stability or instability between the two methods. This is then followed by examining how a point target (i.e., convex mirror) manifests itself before and after orthorectification across all 18 overpasses. Next, we focus our attention on radiometric accuracy concerns for a handful of point target observations. Lastly, we investigate the spectral inconsistencies introduced into the point target spectral radiance after orthorectification.

5.1. Overall Point Target Performance

Figure 11 illustrates the main findings when comparing point and Lambertian targets over multiple overpasses. Even though all the point targets are similar (i.e., mirror-to-mirror variation less than 4%), their standard deviations are significantly greater than the Lambertian targets that span the detector’s dynamic range (i.e., the HSI system was optimized for a 50% reflector). The main limitation of the use of Lambertian targets for radiometric performance is their overall spatial uniformity when compared to point targets. Lambertian targets can isolate an HSI system’s absolute radiometric performance, but the high spatial uniformity masks orthorectification artifacts. The use of point targets provides the best technique to examine the system-level radiometric performance of both the instrument and orthorectification. Figure 11a demonstrates that the radiometric performance over Lambertian targets is less affected by orthorectification errors due to their high spatial uniformity compared to point targets.

Within Figure 11, the general spectral shapes require further explanation. Excluding values at both ends of the wavelength range (i.e., close to 400 nm and 1000 nm), we see a significant trend of worsening measurement dispersion at the bluer wavelengths for point targets. We speculate that the reasoning for this is the interplay between orthorectification and the wavelength-dependent SPSF sharpness (Figure 17) [23]. Depending on how the point target is imaged, the orthorectification struggles with sharper targets compared to targets having a larger spatial extent. This trend continues until the target becomes much larger than the SPSF or, in this comparison, a Lambertian target, where the image-to-image repeatability is small (see Figure 11b). Radiometric errors induced by orthorectification on small targets not only exist for the point targets but will manifest in any small target within the scene. Insight into these observations and the ability to extract radiometric inconsistencies of small targets is only possible with high-contrast point targets (i.e., mirrors). Further wavelength-dependent orthorectification errors are discussed in Section 5.3.

A lesser known effect that may account for the unreal spectral features seen in the point target results in Figure 11a is the potential for polarization sensitivity. As mentioned in Section 2.3, the point targets are convex mirrors coated with protected aluminum. The aluminum layer is a metal thin film and has the potential to reflect polarized light up to the sensor at specific solar zenith angles. Since the HSI system has a reflective holographic grating, there is an inherent grating efficiency for the various polarization states. If the HSI system reimages the point target at various polarization states and orthorectified to a single target, the observed spectral features could potentially exist in the results shown in Figure 11a [15,26].

Figure 12 demonstrates how fixed mounted HSI systems image the ground scene where platform motion significantly impacts the ground sampling. This observation is seen for all the point targets with the exception of a few outliers, to be discussed in Section 5.3, even though the spectral radiance is underrepresented in the unorthorectified image. The main contribution to this over-estimation is orthorectification errors induced by platform motion (e.g., re-imaging the same point target) and interpolation artifacts within the orthorectification process (e.g., pixel values manipulated).

A concluding thought on the results and statements presented in this section relates to the predicted entrance aperture-reaching spectral radiance and the GSD dependence. Equation (3) has an inverse squared dependence on the GSD. This is important to recognize because of the inherent roll/pitch and motion smear from the drone. This physically manifests in a larger GSD compared to the orthorectified pixel. For a predicted entrance aperture-reaching spectral radiance, this would result in an over-predication compared to the observed data. However, Figure 12 displays the reverse: all the observed data are larger than the predicted entrance aperture-reaching spectral radiance. This puts further emphasis on the dependence of the orthorectification process. The sensitivity between the mirrors signal and the GSD is shadowed by the orthorectification and interpolation errors.

5.2. Radiometric Accuracy Concerns

Figure 13 examines two overpasses of the same point target (mirror ID: 12.5MM-4) imaged within 2 min. It has already been demonstrated that the atmosphere and HSI instrument stability cannot explain this drastic increase in perceived energy from the point target. The only difference between the imaged point targets is the platform motion (fixed mounted HSI) and resulting orthorectification.

If the unorthorectified curves (and point target imagery) are examined more closely, we can begin to form definitive conclusions on the reasoning for such drastic difference in the perceived spectral radiance within Figure 13. When the mirror was imaged on the second overpass, the perceived energy was lower than expected, even though the orthorectified signal was accurate according to the prediction. In this instance, the orthorectification properly interpolated the scene to add an adequate amount of energy back that was never physically observed. This is one of the largest contributors to the large measurement dispersion seen in the point targets. There is nothing within the post-processing algorithm that provides enough information to accurately inject the correct amount of energy that was never observed due to inconsistent sampling of the ground during imaging. For the orange dashed curve, the HSI system actually imaged the target or ground location too many times, and the orthorectification cannot realistically dispose of the excessive amount of energy from the point target. This is also observed in Figure 14 as well. This observation provides a missing link within orthorectification algorithms that can be easily seen when imaging high-contrast point targets (i.e., convex mirrors) of a known radiometric signal. If information is not observed or is in excess, orthorectification algorithms can only use IMU/GPS and interpolation schemes to correct the scene.

As mentioned before, orthorectification does not have the correct information to correctly dispose of excess collected energy when a point target is re-imaged. As Figure 14 shows, the point target pairs are similarly affected in the unorthorectified imagery where both record roughly twice the expected mirror signal. However, a difference between the point targets can be seen after orthorectification, where the point targets no longer overlap. Even though the point targets were aligned in the across-track direction, the orthorectification affected the spectral radiance curves differently, further highlighting the volatility in localized regions.

These results demonstrate a clear lack of realism in orthorectifying algorithms for pixel-level details for fixed mounted HSI systems. Multi-modal imaging capabilities and constrained budgets are situations where the radiometric accuracy of the HSI system is sacrificed for payload configuration. At a minimum, convex mirrors provide a low-cost solution for assessing the radiometric performance for any payload configuration. The most optimal mounting solution for HSI systems is a gimbal mount. The reduction in the HS instruments relative motion to the drone will reduce the motion-induced radiometric errors as seen in the results. A gimbal mounting solution will not fully remove all orthorectification errors. However, using the technique illustrated in this paper, the radiometric performance for small targets can be estimated.

5.3. Spectral Inconsistencies

For most HSI applications, the overall signal accuracy has less of an impact on the algorithm performance compared to the spectral shape. For sub-pixel target detection or spectral unmixing algorithms, the reliance on the repeatability of the spectral component is critical to identify spatially unresolved objects. Significant issues can arise within algorithms if the spectral component of small targets is not conserved over all images. Figure 15 and Figure 16 are examples where the primary contribution to spectral distortions related to the point target signal is the orthorectification process and the interpolation scheme.

During the orthorectification process, nearest neighbor or linear interpolation is implemented to construct the scene, but both of these schemes have their issues when encountering a point target. Nearest neighbor interpolation has the issue of replicating or replacing pixels within a scene. For point targets, this can drastically modify the radiometric and spectral integrity due to the inherent SPSF sharpness. Linear interpolation has a lesser effect on the signal level because it has an averaging effect but has significant issues by creating “unphysical” spectra [11]. In addition, both interpolation schemes will have differing performance when HS instrument defects are present (i.e., smile and keystone). HS keystone will produce the worst effects for small spectral targets because pixels of varying spectral quantities will be replicated or replaced and will produce a modified spectrum. To further complicate this problem, the polarization sensitivity of HS instruments will add to spectral inconsistencies.

During imaging, the ground was not sampled adequately and resulted in missing energy from the point target. However, during orthorectification, the linear interpolation derived more energy and spatially modified the point target. The orange curves within Figure 15 exhibit a more complicated orthorectification process that spectrally modified the mirror signal. Again, during imaging, the mirror’s signal was not fully captured by improper sampling of the ground. But instead of adding energy to the mirror’s signal, this is one of the few cases where the orthorectification caused a massive under-prediction of the mirror’s spectral radiance. The main cause is the complete elimination of the pixel highlighted with the red “X” as seen in the left side of the figure. Because of the wavelength-dependent SPSF sharpness, the amount of energy missing after orthorectification affects the shorter wavelengths more. Furthermore, the red arrow (bottom right of figure) indicates a row of pixels that were linearly interpolated but does not fully account for all the missing energy in the blue end of the spectrum.

When entire rows are interpolated (Figure 15), the instrument defects (i.e., keystone) can have negligible effects on the point target’s overall signal because all keystone artifacts are replicated in the interpolation. This is not true when single pixels are interpolated as seen in Figure 16. More importantly, the signal reconstruction will be further impacted if the observed spectral radiance experiences a polarization dependence. The red boxes (right side of figure) highlight pixels that never existed in the unorthorectified image. Since these pixels are a weighted linear combination of their surroundings, the added energy does not combine to realistically replicate the predicted energy defined by the black spectral curve.

Since the mirror’s energy is a summed quantity, any instrument defects should not impact the overall spectral signature, even though single pixels may have a spectrum that has been modified by keystone effects. This is true as long as the ensquared energy for all wavelengths is captured in the bounding box. The important conclusion from this analysis is that single-pixel modification drastically affects the spectral component of small targets compared to entire rows being added. Spectral changes of unknown small targets will have more consequences on an algorithm’s performance because these routines tend to key off the spectral shape. The well-known (solar-like spectrum) ability to predict the radiometric signal from convex mirrors lends itself to the only method that we know of for understanding spectral impacts linked to the HS orthorectification process.

6. Conclusions

In this paper, the radiometric performance assessment on small targets was assessed for a fixed mounted drone-based hyperspectral imaging system. Convex mirrors were used to create point targets that were radiometrically connected to the solar spectrum. Point targets provided the ultimate radiometric test in reconstructing an accurate radiometric signal from orthorectified HS imagery. Results demonstrated the lack of physical realism in the orthorectification process when analyzing point targets. In retrospect, Lambertian targets showed excellent repeatability and self-consistency over multiple overpasses compared to point targets. There was an 18.1% chance that a point target was accurately predicted using radiometric equations defining the spectral radiance of a point target. Major issues originated from both the mounting of HSI systems and the orthorectification process itself. A radiometrically accurate HSI system tends to overestimate the true spectral radiance of small targets based on the nearest-neighbor interpolation scheme used when orthorectifying imagery. Due to the sub-pixel response of the imaging system to a point target, the orthorectification and interpolation can be assessed for deriving radiometrically accurate HS images for the most difficult object, a small target.

A major challenge for drone-based HSI systems is stable along-track sampling (i.e., sampling in the direction of motion). Thus, future work should examine the relationship between orthorectification and along-track sampling over point targets from a linear translation stage. The primary focus of this experiment would be to recreate field conditions similar to drone-based campaigns but restricting the platform motion to the along-track direction. Limiting the platform motion in this direction would provide a technique for extracting the true correlation between orthorectification and small target radiometric performance at different along-track sampling rates. More importantly, different interpolation schemes could be examined to determine the radiometric and spectral integrity for small targets.