// This is to check if a process is running. Used with my bot

#!/bin/sh
# Check if bot is runing then exit otherwise restart
# put in crontab
# 0,10,20,30,40,50 * * * * /path/to/check.sh


MYPATH=/path/to

PID=0

if test -r $MYPATH/bot.pid; then
	PID=$(cat $MYPATH/bot.pid)
fi

if [ 0 -ne $PID ]; then
	running=`ps --pid $PID | grep $PID |wc -l` 

	if [ $running -eq 1 ]; then
		exit 1 
	fi
fi

cd $MYPATH
./check.sh & >/dev/null

// Not so much code here as a record of events and observations
// The summary of countless hours of 'fun' tracking down reasons and solutions for flexbackup not running happily as a cron job.

Flexbackup is a well tried script to automate backup to tape of fixed-disk. It will run in Linux, BSD and Solaris (using gnu findutils [tar and find]) .

It uses perl and shell scripting to elegantly backup data 'sets'.

It uses a configuration file flexbackup.conf stored in /etc

In certain circumstances this will not run cooperatively as a 'cron' job.

This note refers to dealings with an aging Sun system running Solaris. It is considered that these notes will apply equally to many other areas. This is the result of a lot of searching and testing to find a workable solution.

The configuration was tried and tested using the terminal as direct input for commands but failed when run as a 'cron' job.

Interesting elements in my flexbackup.conf are:

##################### /etc/flexbackup.conf ################
$type = 'afio';

$compress = 'gzip'; # one of false/gzip/bzip2/lzop/zip/compress/hardware
$compr_level = '4'; # compression level (1-9) (for gzip/bzip2/lzop/zip)

# Buffering program - to help streaming
$buffer = 'false'; # one of false/buffer/mbuffer

$device = '/mnt/backup'; #the backup raid system on another machine in another office

# I had trouble with this and on an error from cron (it reported that ssh wasn't in the path) I put in the full path...however flexbackup script mistreated that on a content test so I added it as a path below
$remoteshell = 'ssh'; # command for remote shell (rsh/ssh/ssh2)

# I kept this as recording dates as it is useful when they are moved or deleted
$staticfiles = 'false';

$logdir = '/mnt/backup/log/flexbackup_log'; # directory for log files
$comp_log = 'gzip'; # compress log? false/gzip/bzip2/lzop/compress/zip
$staticlogs = 'false'; # static log filenames w/ no date stamp
$prefix = ''; # log files will start with this prefix
$tmpdir = '/mnt/backup/tmp'; # used for temporary refdate files, etc
$stampdir = '/mnt/backup/stamps'; # directory for backup timestamps
$index = '/mnt/backup/log/flexbackup_index'; # DB filename for tape indexes
$keyfile = '00-index-key'; # filename for keyfile if archiving to dir
$sprefix = ''; # stamp files will start with this prefix

# Example: If GNU tar is called "gtar" on your system:
# $path{'sudo'} = '/usr/local/bin/sudo'; # see note below
$path{'tar'} = 'sudo /usr/local/bin/tar';
$path{'pax'} = 'sudo /bin/pax';
#
# Or can be used to "sudo" certain commands. Examples:
$path{'find'} = 'sudo /usr/local/bin/find';
# $path{'dump'} = 'sudo dump';
# $path{'afio'} = 'sudo -u nonrootuser afio';
$path{'afio'} = 'sudo /usr/local/bin/afio';
$path{'ssh'} = '/usr/local/bin/ssh';
#

############################ /etc/sudoers ##############

#additions

Cmnd_Alias FLEXBACKUP = /bin/flexbackup, /bin/pax, /usr/local/bin/find, /usr/local/bin/tar, /bin/gzip, /usr/local/bin/afio

flexbackupuser ALL = NOPASSWD: FLEXBACKUP

############################ end ###################

################### comments and notes ##############

Use of crontab

Check and/or amend the editor otherwise you will get good old ed ... aagh!

$export EDITOR=vi
$sudo crontab -e

By using this command the cron daemon will autmatically be restarted to pickup the amendments.

Initially, knowing that cron disliked output even though it 'might' send it by mail...I used the following command...(for a 1:45pm test after a successful full starting backup from the terminal.

In crontab...

45 13 * * 1-5 /usr/local/bin/sudo /bin/flexbackup -set test_set -level incremental > /dev/null 2>&1

Finally realising problems I redirected output to a file using:

50 13 * * 1-5 /usr/local/bin/sudo /bin/flexbackup -set test_set -level incremental 2>&1 >> /path/to/tester.log

Note that sudo lived in /usr/local/bin.

Long story cut short...

cron operates out of its own (very very limited environment. In other words amongst other things its PATH is strictly limited. For the Sun Solaris this default path is detailed here and usually for all root jobs is restricted to /usr/sbin and /usr/bin.

http://docs.sun.com/app/docs/doc/816-1055/6m7gh31f1?a=view

I tried amending many of the paths on flexbackup.conf including setting these (which failed variously):

DO NOT USE FOR INFORMATION ONLY - THESE DO NOT WORK
$path{'sudo'} = '/usr/local/bin/sudo'; # see note below
$path{'tar'} = '/usr/local/bin/sudo /usr/local/bin/tar';
$path{'find'} = '/usr/local/bin/sudo /usr/local/bin/find';
$path{'afio'} = '/usr/local/bin/sudo /usr/local/bin/afio';
$path{'ssh'} = '/usr/local/bin/ssh';

Ultimately, I was concerned about opening too many additional paths for cron so I simply copied sudo from /usr/local/bin to /usr/bin (ironically I used sudo to copy sudo).

I didn't want to copy over the other binaries as most of them were late additions to the system and in particular would conflict with tar and find. The binaries listed in /usr/local/bin for example were gnu tar and find etc.

I could then delete the path to sudo and successfully created a sequence of backups using cron.

The final cron instruction therefore can also change the location of sudo to /usr/bin/sudo but it exists in both directories now.

For further information.

To list contents (it creates its own output log file in the current directory and rushes a copy to stdout to test your speed reading skills!

flexbackup -list name_of_the_archive.afio-gz

Interestingly this log file is suffixed with the date-time of the enquiry to avoid confusion with other logs.

To extract a single known file:

flexbackup -extract name_of_the_archive.afio-gz -onefile name_of_the_requested_file

...and there it is in the current directory. At last!

Restore?
Not tried yet

full means full/all
differential means only the files changed/added since the last backup (of that set)
incremental means only the files changed/added since the last backup

So differential is ideal for weekly catch all backups and incremental for daily backups.

An idealised makeup is a monthly full, with incrementals everyday then a weekely differential, then delete the incrementals and start them again. Then do another full and rotate the last full to tape or another partition.

To check space used by directories use:

sudo du -sk dir_name

or in a current directory sud du -sk *

(Output is in kb)

I achieved about 50% reduction in size but that obviously depends on numbers of uncompressed items.

I am happy with afio for integrity of individual files. I wanted to try pax but it failed. I never examined the reason.

I hope this helps somebody. It took me absolutely ages to sort out but I really wanted to get flexbackup working completely.

Afio used is version 2.5

This takes a username, password, url and optional time out

require 'net/http'
require 'net/https'
#Usage: username pass urlStr time_out
#

    urlStr = 'http://localhost:3000/cron/cron'
    username = "badname"
    pass = "badpass"
    time_out = 60

    if ARGV[3] != nil
     time_out = ARGV[3].to_i
    end

    if ARGV[2] != nil
     urlStr = ARGV[2]
    end
    
    if ARGV[1] != nil and ARGV[0] != nil
     username = ARGV[0]
     pass = ARGV[1]
    end
    puts urlStr + " user: "+username
    
    url = URI.parse(urlStr)
    use_ssl = url.scheme == 'https'
    req = Net::HTTP::Get.new url.path
    req.basic_auth username, pass
 
    http = Net::HTTP.new(url.host, url.port)
    http.read_timeout=time_out
    if use_ssl
      http.use_ssl = true
    end
    res = http.start { |web| 
      web.request(req) 
    }
    
    puts res.body

// remember to check that the path is correct.
// This is running on a dedicated server on TextDrive

#!/bin/sh

# This file will run a backup of your desired MySQL database and
# remove any backups older than 7 days.
#
# If youOd like to preserve backups for longer than a week, like say 
# 2 weeks, then set the '-mtime' value from '+7' to '+14'.
#

TIME_STAMP=`date "+%Y-%m-%d"`
echo "starting "$0" on" `date`
for db in db1 db2 db3
do
  DB_STAMP=${db}_${TIME_STAMP}
  echo ${DB_STAMP}
  /opt/csw/mysql5/bin/mysqldump --opt --skip-add-locks --user=username --password=password ${db} | gzip > /domains/backups/mysql/${DB_STAMP}.gz
  /opt/csw/bin/mutt -s "mysql  ${TIME_STAMP}"  -a /domains/backups/mysql/${DB_STAMP}.gz someuser@somedomain.com </dev/null
done

cd /domains/backups/mysql/

/usr/bin/find *.gz -mtime +14 -exec rm {} \;
echo "finished "$0" on" `date`

This cron job will create a compressed backup of all the mysql databases under your account. The backup will be stored as "\daily-backup\Mon.gz" - and so forth, one for each day of the week. In this way you will have rotating backups going back seven days.

First, create the "daily-backup" folder under your home directory.

Go into the System - Cron Jobs section in webmin and paste this in as a new cron job (all one line)

/usr/local/bin/mysqldump --skip-opt -uUSERNAME -pPASSWORD --quote-names --complete-insert --extended-insert --quick --compact --lock-tables=false --skip-add-locks --all-databases | gzip > /home/USERNAME/daily-backup/sql-alldb-`date "+%a"`.gz 

Make sure to replace the USERNAME and PASSWORD with your own info.

You can set it up to run on any kind of daily schedule; I have it set to run daily at an early-morning time that I picked randomly.

/usr/local/bin/ruby /home/rsimplicio/web/instiki/instiki.rb --daemon --port 8966 --storage /home/rsimplicio/web/instiki/storage/

To run a php-script as a regular job, you need just point to the php-cli instance with the path to your script:

/usr/local/bin/php /home/username/public_html/scripts/your_script.php

Thanks to Jason for original tip @ http://forum.textdrive.com/viewtopic.php?id=863