Finding interesting things in lots of data

Many large collections of data contain information that is never discovered because the definitions of the objects of interest are many levels of abstraction removed from the level of the raw data. The authors describe their experience with some simple prototyping tools used to find convoys of processes moving through a network of queues from a database consisting of a low-level event trace. Described is the design of an integrated system with which large data sets can be interactively explored in order to locate ill-defined phenomena or objects. Issues related to user interface, language, and system support are discussed. Problems involving wind-tunnel data and weather data are considered insofar as they related to the convoy problem considered