Filenames and Backups

by

A resource I created as part of a group assignment in my research data management course.

File and Folder Naming

Benefits:

  • The first person you share data with is yourself
  • Provides context without having to open the file
  • Finding desired files quickly
  • Distinguishing between similar files
  • Versioning files
  • Easier auditing, collaboration, and reuse
  • Much easier to do it now than redo it later

What to include:

  • Document type (interview, microscope image, questionnaire response; not pdf, spreadsheet—that’s already in your extension)
  • Participant ID, Researcher ID
  • Date collected
  • Version number
  • Project number
  • Department name
  • Any variable/column name in a spreadsheet

Tips:

  • Consistency is key!
  • Document the convention in your README
  • Don’t use punctuation or special characters like: !@#$%^&*()[]{}/?\|,<>.;:
  • Do use one of UPPER, lower, CamelCase or snake_case
  • Follow ISO-8601 when putting dates in your filenames, YYYY-MM-DD or YYYYMMDD, so they are consistent and sort properly
  • For number sequences like participant ID, use leading zeros like 0001 if you’ll have 1000-9999 participants
  • Try to keep it under 30 characters
  • Use _v1 _preprint or similar meaningful name, rather than _final_v2_final

What might you want to use to identify a file without having to open it? If you need to be able to easily find the file for participant 26, then only having the researcher ID and the document type in the filename won’t be enough. But including too many makes the system cumbersome, and naming a new file will be a lot of typing.

The position of fields determines how they’ll sort. If you want to sort by researcher name, and then for each researcher by participant, and then for each participant by date, that’s the order the fields should be in your file or folder name.

Are you more likely to be asked for all the data for participant 26, or all the data collected by research assistant 3? Will you need to batch process files by date or by document type? These questions can help determine how to create your folder hierarchy. For a given project, it’s usually best to have 1-3 levels of folder structure.

Backup Strategies

3-2-1 Rule

You need to have at least: 3 copies of your data, 2 different storage media, and 1 copy at an off-site location.

  1. Copies means that your data are in different places, i.e. not data_v1.dat and data_v1_copy.dat in the same folder. Why? In case you accidentally hit the delete key, you still have the data.
  2. Storage medium is a physical device, like an internal hard drive, external hard drive, USB thumb drive, your co-researcher’s computer, a department file server, or a cloud storage provider. Why? If your own hard drive fails, you still have the data.
  3. Off-site means geographically diverse, somewhere far away from your computer/lab/university. Why? so if your lab burns down or your institution is hit with a ransomware attack, you still have the data.

    Here, Near, Far

    The 3-2-1 rule can be fulfilled by the Here Near Far rule:

    Here – local/working copy

    Near – other local/external copy

    Far – remote copy, stored off-site

    Do not test your backup strategy. Test your restore strategy. On a regular schedule, verify that you can restore data from all of these different locations successfully.