File Systems

User Information Home
Data
Security
Allocations
Consulting
Training

1. Directories
   1. Home Directories
   2. Scratch Directories
   3. /tmp and /var/tmp Directories
   4. Reserved Project Space
2. Permanent File Storage
3. File Transfer Options

---

1. Directories

1.1 Home Directories

Your home directory is the default directory you are placed in when you log on. You should use this space for storing files you want to keep long term such as source code, scripts, input data sets, etc. NCSA HPC systems have a 50GB home directory quota.

The command to see your disk usage and limits is quota. Home directories are backed up daily.

1.2 Scratch Directories

Scratch file systems are intended for short term use and should be considered volatile. The size of scratch file systems varies with the system.

Please note that backups are not performed on the scratch directories. In the event of a disk crash or file purge, files on the scratch directories cannot be recovered. Therefore, you should make sure to back up your files to permanent storage as significant changes are made (at least daily).

• Dell NVIDIA Linux Cluster (forge)

  • Global Scratch Space

    The common scratch filesystem /scratch/users is available for all interactive work. There is a soft link scratch-global in your home directory that points to your scratch directory. For batch jobs, see the section Disk Space for Batch Jobs.

    No scratch disk quotas are currently in place. No guarantees are made as to how much scratch space will be available at any time. Files may be removed at any time should disk space become critically low.

    The purge policy is as follows:

    • Files in the common scratch filesystem (/scratch/users) are removed on the basis of size and time since the last access:

      File Size    Removed after
      >= 10 GB            4 days
      <  10 GB           14 days 
      

      Please do not attempt to circumvent this removal scheme (e.g., with touch). Such attempts may result in the loss of access to the scratch file systems.

    • Files in the batch scratch filesystem (/scratch/batch) on Forge may be purged as soon as the batch job that it is associated with completes. Users should use the saveafterjob utility for automated, guaranteed saving of files from batch jobs.

  1.3 /tmp and /var/tmp Directories

  The /tmp, /usr/tmp, and /var/tmp directories are intended for temporary files that are used during the execution of a process or job. Please do not use these directories for storage of user files. Files placed in /tmp, /usr/tmp, or /var/tmp may be purged at any time.

  1.4 Reserved Project Space

  NCSA has reserved project space available on the HPC systems. They are intended for users who require larger amounts of scratch space than exist in the normal scratch file systems for short durations. These directories are intended for short term use and are not backed up. Contact consult@ncsa.illinois.edu for requesting space.

  Please include your name, project (PSN), preferred start date, duration of need, approximate disk space needed, and a short description of the project. Include reasons why regular scratch space is insufficient for your needs.

  2. Permanent File Storage

  Permanent storage of your files is available using the NCSA archival storage system (MSS). You have read-only access to your MSS account for 4 months after your last NCSA HPC account has been deleted.

  The NCSA MSS Page has additional information on using MSS. See the Overview section in the MSS User Guide for user and project quota information.

  3. File Transfer Options

  A variety of methods are available for transferring files between computer systems. These are outlined, with examples, here: NCSA Data Transfers