The Semantic Gap: An Exploration of User and Computer Perspectives in Earth Observation Images

Research on the semantic gap has considered differences between user and computer image interpretations and proposed methods to bridge it. These methods have been verified by comparing results to reference data or by measuring the degree of user acceptance. Although these methods result in a narrower semantic gap between computers and users, the resulting model for a specific user and search goal may still not be satisfactory to other users. Through an image annotation task with users, we find that this discrepancy is caused by the subjective biases present in the bridging methods, which we refer to as the “linguistic semantic gap.” Based on our findings, efforts to bridge the semantic gap should include different user perspectives to compensate the individual subjective biases, by increasing the diversity of data sets used in the domain. Moreover, models derived from proposed bridging methods could be stored and further used by other systems.