Against conventional wisdom: Longitudinal inference for pattern recognition in remote sensing

In response to Democratization of Imagery, a recent leading theme in the scientific community, we discuss a persistent imaging experiment dataset, which is being considered for public release in a foreseeable future, and present our observations analyzing a subset of the dataset. The experiment is a long-term collaborative effort among the Army Research Laboratory, Army Armament RDEC, and Air Force Institute of Technology that focuses on the collection and exploitation of longwave infrared (LWIR) hyperspectral and polarimetric imagery. In this paper, we emphasize the inherent challenges associated with using remotely sensed LWIR hyperspectral imagery for material recognition, and argue that the idealized data assumptions often made by the state of the art methods are too restrictive for real operational scenarios. We treat LWIR hyperspectral imagery for the first time as Longitudinal Data and aim at proposing a more realistic framework for material recognition as a function of spectral evolution over time. The defining characteristic of a longitudinal study is that objects are measured repeatedly through time and, as a result, data are dependent. This is in contrast to cross-sectional studies in which the outcomes of a specific event are observed by randomly sampling from a large population of relevant objects, where data are assumed independent. The scientific community generally assumes the problem of object recognition to be cross-sectional. We argue that, as data evolve over a full diurnal cycle, pattern recognition problems are longitudinal in nature and that by applying this knowledge it may lead to better algorithms.