Scrubber: don’t lose your files!

On October 19, we will turn on the STF file scrubber which will remove files on hyak in /gscratch/stf that are older than 30 days old. Here is some information that will help you prepare for the scrubber launch date and live with the scrubber once it’s here:

  • From October 2-18, the file scrubber will run every day in “dry run” mode, producing a list of files that would be removed and storing the output in /gscratch/stf/scrubbed.txt. You should look at this file to see what files will be deleted after October 19. It’s a long list of files so you could narrow it down by using something like grep "dir_name" scrubbed.txt, where dir_name is some directory that contains your files.
  • If you no longer need these files, please go ahead and delete them. This will make it easier for others to navigate the scrubbed.txt file.
  • If you do want to keep these files, you have a few options:
    • For your own small scripts or personal files (e.g. login scripts, ssh keys, or your .vimrc), you can put these in your home folder in $HOME.
    • For software that you need in order to do your research (e.g. the fftw library or gromacs), you should put these files into the /sw/contrib directory. In this way, other users can benefit from the hard work you put into building these applications on Hyak. You might even find that your application is already there.
    • For results or scientific data that you need to keep around for longer than a month, please consult the Hyak wiki page on managing your files. Your options include transferring files to your own machine or using the lolo remote file system.

All of these tactics are meant to help you prepare for the scrubber launch date but they are also good habits for working on Hyak after the file scrubber has been implemented.

One final note: please do not try to cheat the scrubber. I’m sure you can all think of different ways to fool the scrubber but it’s there to ensure that we’re being good Hyak citizens. And there are so many good legitimate alternatives for managing your files that it really isn’t worth it to try to work around the system. Any STF user who manipulates files in /gscratch/stf in order to deceive the file scrubber will have their STF account suspended. We’ve got a great group of students here so I know that won’t be a problem. Happy computing.