High-performance distributed digital libraries: building the Interspace on the grid

The Net of the 21st Century will radically transform interaction with knowledge. Users will navigate in the Interspace, across logical spaces of semantic indexes, rather than in the Internet, across physical networks of computer servers. Correlation across indexed collections is the most important feature of this infrastructure. Over ten years of research, the author has developed scalable technology for generating the necessary semantic indexes. Construction of large scale models of the Interspace is feasible now under controlled laboratory conditions. Community repositories for entire scientific disciplines have been constructed using supercomputer simulations on millions of documents. A model Interspace is a set of community repositories, interconnected by concept switching networks to support information analysis across subject domains. CANIS has constructed several model testbeds with increasingly better infrastructure technology. We propose a PACI Interspace for the NSF flagship efforts of the HPDC community. The Interspace would provide concept switching for the users while the Grid would provide object switching for the sources