Astronomical tables, 2-D indexing, and fuzzy-joins

Tabular datasets of up to 10⁹ rows can be produced by high-energy telescopes which return the properties of every detected photon. Catalogues of celestial objects are also simple tabular datasets which are starting to reach comparable sizes and present similar problems. The operations to be performed on such tables can often be described in relational database terms, but in practice special-purpose software is mostly necessary. Some of the problems which arise when handling these tables are discussed in this paper, in particular the need for two-dimensional indexing and the fuzzy join, and some possible solutions are explored