Making Site Backups

Site backups proceed in two phases. First, archives are made of relevant files on the server, then these archives are downloaded to an external harddrive. At the time of writing (2013-06-23), this is done remotely through the use of Python's fabric library.

Server-side backups

Site backups are current done through shell scripts located in the main directory.

At the time of writing, the following scripts exist:

  1. backup_aclwiki.sh
  2. backup_adminwiki.sh
  3. backup_execwiki.sh
  4. backup_newsite.sh
  5. backup_nmlwiki.sh
  6. backup_portal.sh

The scripts vary by target, but basically they all operate by creating timestamped archives of designated folders and/or the output of a database dump.

Fabric and Client-side downloads

Client-side, python fabric scripts are used to activate the shell scripts remotely, download their output, and then erase the backup files from the server (so as not to clutter our webspace).

fabric scripts are run by typing fab {commandname} at the commandline. The actual commands are stored in a file called (by convention) fabfile.py; each function definition in such a file words as a command.

More details are available on the fabric documentation page

A sample fabfile.py:

from fabric.api import *
from datetime import date

env.hosts = ['www.aclweb.org']
env.user = 'u36925139'
env.password = '{current password}'
env.port = 22

def backupportal():
  datestring = date.today().strftime('%Y%m%d')
  backupfile = "backup-acl_portal-{datestring}.tar".format(datestring=datestring)
  local("ssh www.aclweb.org -l u36925139 \"./backup_portal.sh\"")
  local("scp u36925139 [at] www.aclweb.org:~/{backupfile} /Users/vimder/ACL/backups/".format(backupfile=backupfile))

def backupnewsite():
  datestring = date.today().strftime('%Y%m%d')
  backupfile = "backup-acl_newsite-{datestring}.tar.gz".format(datestring=datestring)
  local("ssh www.aclweb.org -l u36925139 \"./backup_newsite.sh\"")
  local("scp u36925139 [at] www.aclweb.org:~/{backupfile} /Users/vimder/ACL/backups/".format(backupfile=backupfile))

def backupwiki(wiki='aclwiki'):
  datestring = date.today().strftime('%Y%m%d')
  backupfile = "backup-acl_{wiki}-{datestring}.tar".format(datestring=datestring, wiki=wiki)
  local("ssh www.aclweb.org -l u36925139 \"./backup_{wiki}.sh\"".format(wiki=wiki))
  local("scp u36925139 [at] www.aclweb.org:~/{backupfile} /Users/vimder/ACL/backups/".format(backupfile=backupfile))

def rmcurrent():
  datestring = date.today().strftime('%Y%m%d')
  backupfile = "backup-acl_portal-{datestring}.tar".format(datestring=datestring)
  local("ssh www.aclweb.org -l u36925139 \"rm {backupfile}\"".format(backupfile=backupfile))

def rmcurrentnewsite():
  datestring = date.today().strftime('%Y%m%d')
  backupfile = "backup-acl_newsite-{datestring}.tar.gz".format(datestring=datestring)
  local("ssh www.aclweb.org -l u36925139 \"rm {backupfile}\"".format(backupfile=backupfile))

def rmcurrentwiki(wiki='aclwiki'):
  datestring = date.today().strftime('%Y%m%d')
  backupfile = "backup-acl_{wiki}-{datestring}.tar".format(datestring=datestring, wiki=wiki)
  local("ssh www.aclweb.org -l u36925139 \"rm {backupfile}\"".format(backupfile=backupfile))

def downloadbackup():
  backupportal()
  rmcurrent()

def downloadnewsite():
  backupnewsite()
  rmcurrentnewsite()

def downloadwikibackup(wiki='aclwiki'):
  backupwiki(wiki=wiki)
  rmcurrentwiki(wiki=wiki)

def backupsite():
  message = "Backing up {target}"
  print message.format(target="portal")
  downloadbackup()
  downloadnewsite()
  wikis = ['aclwiki', 'execwiki', 'nmlwiki', 'adminwiki']
  for wiki in wikis:
    print message.format(target=wiki)
    downloadwikibackup(wiki)