Page History

Backup and Restore of Thalia Clusters to the Eurydice backup server.

Setup of Remote Server to be backed up.

1.Login to remote server (part of Thalia cluster) as root user.

2.Create needed directories

    mkdir /root/.ssh
    mkdir /home/rmtbckup

3.Create ssh key

Wiki Markup    ssh-keygen \ -t rsa \ -b 1024 \
    [press return until you are back to command line - do not use encrypted private keys, as this will cause the cron job to fail\]

4.Copy ssh key to eurydice.mit.edu

    scp /root/.ssh/id_rsa.pub \
    rmtbckup@eurydice.mit.edu:/home/rmtbckup/.ssh/`hostname`-id_rsa.pubunmigrated-wiki-markup

5.Login to eurydice as rmtbckup  \as rmtbckup  [note, this should be done from another thalia system, as Ubuntu does not currently have working Kerberos configurations\]

6.Add public key to authorized key lists and deleted from eurydice

     1.On Eurydice:

Wiki Markupcd          cd /home/rmtbckup/.ssh
         cat \ [remote hostname\]-id_rsa.pub >> authorized_keys
         rm \ [remote hostname\]-id_rsa.pub

     2.On remote server:

         cd /root/.ssh
         cat id_rsa.pub >> authorized_keys

7.Install the thaliazinc-backup.py script into /root on the remote server, and add the following line to /var/spool/cron/root:

    30 3 * * * /root/thaliazinc-backup.py >> /loc/logs/thaliazinc-backup.log 2>&1 &

8.Test backup per manual backup procedures (below)

Procedures to manually backup Thalia clusters.

1.Hypothetically, only one stack in a thalia cluster needs to be backed up, however, this has not been tested.

...

3.Login to the MySQL server of the Thalia cluster as root, and run the backup script:

    /root/thaliazinc-baclupbackup.py

        Alternatively, you could backup the database with the myslqdump command:

    mysqldump -single-transaction -q -B "dbname" -r /home/rmtbckup/`hostname`-db`date \
    %F+%H+$M`.sql

4.Login to Alfresco server of the Thalia cluster as root, and run the backup script:

    /root/thaliazinc-backup.py

        Alternatively, you could backup the filestore with the tar command:

Wiki Markuptar \    tar -czf /home/rmtbckup/`hostname`-file-`date \ +%F-%H-%M`.tgz /home/\[alf_home\]/alf_data/\*

5.Use scp to transfer backup files to eurydice:

scp /home/rmtbckup/* rmtbckup@eurydice.mit.edu:/home/rmtbckup

6.Login to Eurydice as rmtbckup and check that the files are there.

76.Delete old files from remote servers, do this for both the MySQL and Alfresco serversLet files remain on the local server for capture onto Tape via Tivoli.

Procedures to restore Thalia cluster from backup:

1. Use scp to transfer files from Eurydice to the remote servers in the cluster.  Since Kerberos does not function well from Ubuntu, you will need to login to a Thalia server, then into Eurydice as user rmtbkcup. The files with 'db' in their names are the database backups and the files with 'file' in their names are the filestore (Alfresco) backups.  Match hostname at the beginning of the file to the hostname of the remote server. 

2.Login to the top level Thalia servers in the cluster, and stop Thalia.  Currently, this is isda-thalia5 and isda-thalia8.

    /etc/init.d/web stop

3.Login to the Alfresco server in the cluster, and stop Alfresco.

    /etc/init.d/web stop

4.Clear the filestore.

Wiki Markuprm \-rf /home/\[alf_home\]    rm -rf  $ALF_HOME/alf_data

5.Untar the file backup:

Wiki Markuptar \-xvzf     tar -xjvf /home/rmtbckup/`hostname`-file-\[date of backup\].tgz /home/\[alf_home\]bz2 $ALF_HOME/alf_data

6.Login to the MySQL server, and dump the database.  The database name is available in the alfresco/extensions directory in the custom-db-connections.properties file.

Wiki Markupmysql drop database \    mysql
    drop database [db.name\];
    revoke all privileges, grant option from \ [db.username\]@*;
    drop user \ [db.username\]@*;

7.Source the sql backup, and grant privledges to the Alfresco user.  See custom-db-connections.properites.

Wiki Markupsource     source /home/rmtbckup/\[hostname\]-db[db.name]-\[date of backup\].sql;
    grant all on \ [db.name\].\* to \ [db.username\]@'localhost' identified by \ [db.password\] with
    grant option;
    grant all on \ [db.name\].\* to \ [db.username\]@'localhost.localdomain' identified by \ [db.password\] with grant option;
    grant all on \ [db.name\].\* to \ [db.username\]@\[IP of Alfresco server\] identified by \ [db.password\] with grant option;
    grant all on \ [db.name\].\* to \ [db.username\]@\[hostname of Alfresco server\] identified by \ [db.password\] with grant option;

8.Login to Alfresco server and restart Alfresco.

    /etc/init.d/web start

9.Login to Thalia server and restart Thalia.

    /etc/init.d/web start

10.Test cluster functionality.

backup schedule:

we expect to keep backups for 3 years on tape, 6 months on al-dente, and 2 weeks on the local system. Full backup every Monday morning, and incremental daily the rest of the week.