Input data

Each cluster user can use the ~ as a storage of the input and output data for experiments. Please note, that the /home partition is an NFS share on compute nodes. Because of this fact each time an experiment is run all input data must be copied again from the login node.

Alternatively, most of the compute nodes have a local partition (or two), available for each user. Any data can be copied to this directories (using an interactive session). This approach tries to minimize the network load and speed up experiments.

Warning

Please remember that the /home directory is acessible through the Faculty’s network. Running a job that requires a large input dataset twice, will cause downloading this dataset twice. When using even moderatly big input data, please consider using /storage_type_N filesystems.

/storage_type_N directories

This directories (N = 1, 2, ...) are disk partitions local to a compute node. They are available to all users and no quota is enforced. This filesystem is not synced with any other filesystem and has multiple uses, for example:

  • is freely available during interactive sessions,

  • can fit large input data, virtual environments, etc. created during interactive sessions,

  • is suitable for any kind of results, if they are larger than default limits imposed on the /home filesystem (but not only),

  • any data can be copied outside the cluster using scp, rsync, etc.; there is no automatic system configured to transfer or delete this data. Please use an interactive session to manage files.

But please be considerate of other users, do not use too much space and delete unnecessary data. Please use --nodelist to select a specific node.

Note

As this filesystem is available to all users, a common sense and awareness of others should be exhibited by all users of the cluster.

  • One should delete redundant files.

  • There is probably no need for several copies of the same data.