There is nothing that should prevent you from running a script that creates multiple simultaneous connections to HPSS. The HPSS system administrator recommends that you should not create more than 1 or 2 connections at a time. Every time you introduce a new instance, the performance of the overall system is degraded.
Because HSI is a third-party package, clients may be available for your system; however, NICS currently supports access to HPSS only through HSI clients on Darter and Nautilus.
If you log into Darter or Nautilus using your passcode from your OTP token, you can run HSI without entering your passcode each time. You can also run batch scripts that use HSI in the "hpss" queue. If you logged using GSI authentication you will be prompted for your passcode each time you use HSI.
HPSS performance is greatly improved when the transfer size is between 8 GB and 256 GB. For that reason, users with large numbers of relatively small files should combine those files into one or a few 8 GB to 256 GB files and then transfer the larger files. The files can be combined with
tar on the HPC system, or they can be created on the fly with a command similar to
tar cvf some_dir -|hsi put - : somedir.tar. This command will
tar all files in the
some_dir subdirectory into a file named
somedir.tar on HPSS.
Users may access HPSS from any NICS high-performance computing (HPC) system with the Hierarchical Storage Interface (HSI) utility. An OTP token is required upon entry. Access to HPSS is enabled by typing the command
hsi in your linux environment. To exit, simply type