How I backup my data
Digital data is easy to copy and also easy to lose, due to (usually sudden) failure of electronic devices. A good habit is to backup regularly and in different locations. This post is about my solution to data backup.
Classification of digital data
Digital data can be classified according to their importance and sizes, and the following lists some of the categories based on their importance.
- Data that can be downloaded online
- Papers and e-books collected from various sources
- Personal photos and videos
- Experimental data and code
- Personal documents and writings
Data that can be downloaded online is the lease important, since they can be easily accessed online and even it’s lost, it can be easily found online. Papers and e-books (or other resources) that are collected from various sources and clearly organized are more important. Some resources are hard to find, and once it is found, it’s better to keep it safe and have multiple copies. Personal photos and videos are even more important, to keep a record of ones’s life and happy memories.
The most important data are experimental data and code (related to work), and personal documents and writings (personal life). Experimental data can sometimes be very large, but code and personal documents and writings (and notes) are usually in plain text format and are far smaller in size. My solution to data backup is based on the importance of data and their sizes.
My solution to data backup
Some say that it’s best to have two local backups, one online backup and one backup in another place. Of course, it’s not always possible to have them all. My Macbook Pro is the centre of all my data and the data are usually backup using Time Machine to a hard drive. I also have a SSD drive, which keeps a backup of large files, including paper and documents collected online, and personal photos and videos. Therefore, I have three local backups in total.
For experimental data and code, in addition to local backups, I also upload the data to Onedrive, with the service provided by my organization. It’s good for large data and very handy for sharing data with colleagues. I also use online backup for my notes. I previous use Nutstore to backup the writings and notes, but now it simply keeps another online copy for my notes and writings. I use rsync to sync these data to online locations. For example, my notes are synchronized to Onedrive using the following command.
rsync -av --exclude-from '~/Documents/notes/ignore.txt' ~/Documents/notes/ ~/Documents/OneDrive/notes
Some of my code are in github, and some data and scripts for my papers are uploaded to OSF. But these are used for sharing instead of backup.
I use Syncthing to backup my personal documents and writings. Syncthing is a private and secure tool which synchronizes data among multiple devices. It is cross-platform and perfect for personal documents, writings and notes, which are usually small in size. I use Syncthing to synchronize these data across my MBP, my phone and the Raspberry Pi in my office. Syncthing can also backup the photos I take with my phone to my MBP.
My backup can be summarized as below,