Text retrieval on the AMT DAP

Describes the use of massively parallel computing (in particular, the AMT DAP series) for word based text document retrieval. Wider utilisation of ever larger databases requires higher throughputs of queries scanning larger volumes of data. Conventional systems cannot cope with thousands of queries/minute covering tens of GBytes of data; a single query term may invoke a million hits, and even for a simple Boolean query, merging such lists looking for hit documents is too slow. However, implementing suitable algorithms on a massively parallel computer is achieving two orders of magnitude improvement in both speed and cost effectiveness. The system being implemented initially is for a specific requirement, but the techniques are widely applicable