[Bio-Linux] Cron and Qiime
Tim Booth
tbooth at ceh.ac.uk
Mon Oct 4 09:53:42 EDT 2010
Hi All,
I'm back from hols and have taken a look. Problems like this remind me
why I should always write scripts like this in Perl!
Mike - can you try going back to the original script and simply changing
the troublesome line 33 as follows:
snip>>>
#What to backup
backup_filesystems=(`echo ${FILESYSTEMS:="/ /home /var /usr"}`)
<<<snip
You can also uncomment the DEBUG bit on line 103 if you want to see what
it is doing.
If this works I'll update the package with the change (unless someone
can suggest a neater way than the use of echo + backticks).
Cheers,
TIM
On Mon, 2010-10-04 at 14:34 +0100, Cox, Michael J wrote:
> Hi Tony P,
>
> Sorry, I did uncomment those lines and rerun backup, but it didn't
> return anything different from before. I'm pretty sure that I've
> uncommented the right part of the script (there's only one debug
> line), but here is a full copy of my backup script as it now stands.
>
> Tony T, is the answer to create a file at /etc/default/backup with the
> config details that you list?
>
> Thanks Mike
>
> #!/bin/bash
>
> #Fail on errors!
> set -e
> set -u
>
> #######################################################################
> # /etc/cron.daily/backup - a script to backup one hd to #
> # another on the same machine, using dump #
> # (see the dump man page for details, and #
> # http://envgen.nox.ac.uk/envgen/software/archives/000501.html#auto #
> # #
> # Written by MIT, 20/6/2002 #
> # Modified by Dan Swan for Bio-Linux 2.0 17/3/2003 #
> # Updated for Bio-Linux 5 by Tim Booth 1/8/2008 #
> # Updated to perform Quarterly ETOH backups by Tony Travis 17/7/2009 #
> # Updated to dump ext4 fileystems by Tony Travis 19/5/2010 #
> #######################################################################
>
> # Source configuration file
> CONFIG=/etc/default/backup
> if [ -r $CONFIG ]; then
> . $CONFIG
> fi
>
> #Use basic or ETOH mode?
> backup_mode=${MODE:="etoh"}
>
> #Where to backup
> backup_destination=${DESTINATION:="/backups"}
>
> #What to backup
> backup_filesystems=(${FILESYSTEMS:="/home" "/var" "/usr"})
>
> #Compression scheme. For large drives bzip2 is probably too slow.
> compression_mode=${COMPRESSION:="z"}
>
> # get the date
> day=`date +%a`
> month=`date +%b`
> dom=`date +%d`
>
> #Dump level can be set on command line or by today's date.
> if [ -n "$*" ]; then
> level="$1"
> elif [ "$backup_mode" == "etoh" ] ; then
>
> # Quarterly Enhanced Towers of Hanoi
> if [ $day == "Sun" ]; then
> if [ $dom -le 7 ]; then
> case $month in
> "Jan" | "Apr" | "Jul" | "Oct") level=0 ;;
> *) level=1 ;;
> esac
> elif [ $dom -le 14 ]; then
> level=3
> elif [ $dom -le 21 ]; then
> level=2
> elif [ $dom -le 28 ]; then
> level=4
> elif [ $dom -le 31 ]; then
> level=3
> fi
> else
> case $day in
> "Mon") level=6 ;;
> "Tue") level=5 ;;
> "Wed") level=8 ;;
> "Thu") level=7 ;;
> "Fri") level=9 ;;
> "Sat") level=8 ;;
> esac
> fi
> elif [ "$backup_mode" == "basic" ] ; then
> # dump is 9 any day but Sunday
> if [[ $day == "Sun" ]]; then
> level=0
> else
> level=9
> fi
> else
> echo "ERROR: Unknown backup mode and no level specified"
> exit 1
> fi
>
> # dirs to back up - used to be:
> #backups=(hda1 hda2 hda5 hda6)
> #But now I need to be a bit more sophisticated.
> #List filesystems to back up and use mount to find where they are.
> backups=()
>
> for fs in "${backup_filesystems[@]}" ; do
> device=`mount | egrep "[^[:space:]]+ on $fs type ext[234]" | awk '{print $1}'`
>
> if [ "$device" != "" ]; then
> backups[${#backups[@]}]=$device
> fi
> done
>
> #DEBUG
> echo ${backups[@]} $level $compression_mode
> exit
>
> # if backup drive is already mounted then try to unmount it first.
> # if this is in use the step will fail and exit
> /bin/mount | grep -q "/backups" && /bin/umount /backups 2>/dev/null
>
> # mount necessary drive - must be listed
> # in /etc/fstab in order to be mounted
> /bin/mount "$backup_destination" 2>/dev/null || exit 0
>
> # dump each directory, one at a time
> for item in "${backups[@]}"; do
> basename=`basename $item`
>
> #sync or swim
> sync
> /sbin/dump -"$level" -"$compression_mode" -n -u -f "$backup_destination/$basename.bak.$level" "$item"
> done
>
> # umount drive
> /bin/umount "$backup_destination"
>
>
> On 4 Oct 2010, at 12:00, <bio-linux-request at nebclists.nerc.ac.uk> wrote:
>
> Send Bio-Linux mailing list submissions to
> bio-linux at nebclists.nerc.ac.uk
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux
> or, via email, send a message with subject or body 'help' to
> bio-linux-request at nebclists.nerc.ac.uk
>
> You can reach the person managing the list at
> bio-linux-owner at nebclists.nerc.ac.uk
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bio-Linux digest..."
>
>
> Today's Topics:
>
> 1. Re: Cron and Qiime (Tony Travis)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 03 Oct 2010 16:31:49 +0100
> From: Tony Travis <a.travis at abdn.ac.uk>
> Subject: Re: [Bio-Linux] Cron and Qiime
> To: bio-linux at nebclists.nerc.ac.uk
> Message-ID: <4CA8A1E5.6040006 at abdn.ac.uk>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 30/09/10 15:12, Anthony Pemberton wrote:
> Mike,
>
> OK cron is running. This looks like the problem I encountered with the script /etc/cron.daily/backup - under
> #What to backup
> backup_filesystems=(${FILESYSTEMS:="/home" "/home1"})
>
> Note the use of the double quotes around the partitions you want to backup (these are the mount points). If I remember correctly, the quotes were missing and are now required. I think that is why the script is failing. The default config is a "red-herring" because what the script is saying is: if the file exists read it, if not continue.
>
> The way to quickly test the script after editing is to run it from the command line eg.
>
> % sudo -i
> # cd /etc/cron.daily
> # ./backup
>
> and see what happens. Use of echo statements in shell scripts like this helps to debug,
>
> Hello, Tony.
>
> Tim added my changes to bio-linux-backups, but I have also noticed
> problems if an "/etc/default/backup" file is absent. Sorry, I should
> have posted something to the list.
>
> I use these defaults (from Bio-Linux 5) for our NBX's:
>
> # @(#)/etc/default/backup 2009-07-17 A.J.Travis
>
> #
> # Bio-Linux 5.0 backup configuration
> #
>
> # Where to backup
> #DESTINATION="/backups"
>
> # What to backup
> #FILESYSTEMS="/ /home /var /usr"
> FILESYSTEMS="/ /home"
>
> # Compression scheme. For large drives bzip2 is probably too slow.
> #COMPRESSION="z"
>
> HTH,
>
> Tony.
> --
> Dr. A.J.Travis, University of Aberdeen, Rowett Institute of Nutrition
> and Health, Greenburn Road, Bucksburn, Aberdeen AB21 9SB, Scotland, UK
> tel +44(0)1224 712751, fax +44(0)1224 716687, http://www.rowett.ac.uk
> mailto:a.travis at abdn.ac.uk, http://bioinformatics.rri.sari.ac.uk/~ajt
>
>
> ------------------------------
>
> _______________________________________________
> Bio-Linux mailing list
> Bio-Linux at nebclists.nerc.ac.uk
> http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux
>
>
> End of Bio-Linux Digest, Vol 36, Issue 3
> ****************************************
>
> Mike Cox
>
> Research Associate
> Centre for Respiratory Infection
> National Heart Lung Institute
> Royal Brompton Campus
> Guy Scadding Building
> London SW3 6LY
>
> michael.cox1 at imperial.ac.uk
> http://www1.imperial.ac.uk/medicine/people/michael.cox1/
>
>
>
>
> _______________________________________________
> Bio-Linux mailing list
> Bio-Linux at nebclists.nerc.ac.uk
> http://nebclists.nerc.ac.uk/mailman/listinfo/bio-linux
--
Tim Booth <tbooth at ceh.ac.uk>
NERC Environmental Bioinformatics Centre
Centre for Ecology and Hydrology
Maclean Bldg, Benson Lane
Crowmarsh Gifford
Wallingford, England
OX10 8BB
+44 1491 69 2705
--
This message (and any attachments) is for the recipient only. NERC
is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make may be disclosed by NERC unless
it is exempt from release under the Act. Any material supplied to
NERC may be stored in an electronic records management system.
More information about the Bio-linux-list
mailing list