Input data

Each cluster user can use the ~ as a storage of the input and output data for experiments. Please note, that the /home partition is an NFS share on compute nodes. Because of this fact each time an experiment is run all input data must be copied again from the login node.

Alternatively, most of the compute nodes have a local partition (or two), available for each user. Any data can be copied to this directories (using an interactive session). This approach tries to minimize the network load and speed up experiments.

Warning

Please remember that the /home directory is acessible through the Faculty’s network. Running a job that requires a large input dataset twice, will cause downloading this dataset twice. When using even moderatly big input data, please consider using /local_storage_N filesystems.

/local_storage_N directories

This directories (N = 1, 2, ...) are available to all users and no quota is enforced. Most of the servers have this directory, please consult Compute nodes. This filesystem is not synced with any other filesystem and has multiple uses, for example:

  • is freely available during interactive sessions,

  • can fit large input data, virtual environments, etc. created during interactive sessions,

  • is suitable for any kind of results, if they are larger than default limits imposed on the /home filesystem (but not only),

  • any data can be copied outside the cluster using scp, rsync, etc.; there is no automatic system configured to transfer or delete this data. Please use an interactive session to manage files.

But please be considerate of other users, do not use too much space and delete unnecessary data. Please use --nodelist to select a specific node.

Note

As this filesystem is available to all users, a common sense and awareness of others should be exhibited by all users of the cluster.

  • One should delete redundant files.

  • There is no need for several copies of the same data, please work together if there is such possibility.