Data file management in the DIII-D data acquisition and analysis computer systems

DIII-D is a large tokamak plasma physics and fusion energy research experiment funded by the US Department of Energy. Each shot of the experiment results in data files containing between 20 and 30 Mb of data. These shots occur about once every 10 min, with 40 to 50 shots per operating day. Over 1.2 GB have been acquired in one daily session. Most of this data is acquired by MODCOMP Classic computers and is transferred through a Network Systems Hyperchannel to the DIII-D DEC VAX cluster system which is connected by Ethernet to the user service center DEC VAX cluster system. Some other data is acquired by local MicroVAX-based plasma diagnostic systems and is transferred by DECnet to the DIII-D cluster. A substantial part of these VAX cluster systems is devoted to handling the large data files so as to maintain availability of the data for users, provide for shot archiving and shot restoration capabilities, and at the same time allow for new data to be received into the systems. Many of these tasks are carried out in near real time in sequence with a tokamak shot while other tasks are performed periodically throughout operations or during off hours. These tasks include disk-space management, data archiving to 6250 and/or 8-mm tape drives, data-file migration from the DIII-D cluster to the user service center cluster, data-file compression, and network-wide data-file access