Automation using CRON

This is a very basic introduction to automating administration tasks in unix.If you ever noticed yourself doing the same thing daily, or even weekly then stop and think about how you can automate it. Think of all the things you'd like to do but don't have to time to do daily, weekly. There's no reason why most tasks can't be automated. I will give some examples which are practical and very useful, which could be changed and applied to do most tasks.

1. Using crontab to execute a shell script nightly.
2. Automating network access using ssh and keys.
3. Backing up a mysql database nightly.
4. Doing a remote action based on a real time event.

Please note before I begin any of this, some of what's explained below may be against your sites security policy, however it's also worth noting that risk management should take into account limited network automation tasks.

1 - Using crontab to execute a shell script nightly

This would be a very common situation ,where at a certain time, each minute/hour/day/month a task is performed. The crontab format is quite simple, for example an entry like

01 1 * * * /bin/sh/home/davidd/bin/somescript.sh

will execute /home/davidd/bin/somescript.sh at 1:01 each night.

The format at the beginning is ( minute hour day month weekday ( where 0 = Sunday)). Generally I call most things in a script, so I can just add things in without changing the crontab, also I usually mail all the results back to myself, while it's good to automate tasks, it's ever better than you can see the output every morning.

2 - Automating network access using ssh and key

One of the best tools available to you, is using role accounts with ssh and blank keys. While you have to be very careful the access which this user has, it's a very useful roll, it can copy files remotely and execute remote commands very easily. In short I would advise you to create a user called _backup (the _ because I generally use this for all system accounts which humans don't generally use). Once you have added the user, su - backup and run;

ssh-keygen -t dsa

when prompted for the directory to save to, just hit enter, once prompted for the passphrase also hit enter and do not enter any passphase, you should then get your keys in the .ssh directory of the users home directory. I would advise you to run chmod 700 on the users home directory, so no other users on the system can see files contained in there.

Now we'll take a situation of copying a file from hosta to hostb, totally automated. These next steps will have to be added on any machine you want to communicate with. Now on hostb add a user called _backup and;

su - backup ; mkdir .ssh

this is to create a .ssh directory where we can copy the public key. Once this is done, from hosta run, scp .ssh/id_rsa.pub hostb:~/.ssh/authorized_keys , now from hosta type ssh hostb , then accept the fingerprint of hostb and you should be logged in without being prompted for a password. Now that the key is accepted and stored, you should be able to ssh, or scp without being asked for a password. Note that the commands can only be run from hosta right now. All other machines only have the public key, not the private key (which is .ssh/id_rsa).

To be able to start this from another host, you'd have to copy the .ssh/id_rsa to another server. Now that you can scp (to copy files) and ssh (to execute commands on the remote machine and have the results returned on the tty), we should move onto the next section, which makes a practical use of this for backup purposes.

3 - Backing up a mysql database nightly.

This section will draw on knowledge from the other two, you'll be using a shell script, called by cron, using ssh with keys for network communication. First we need to know what to put in the script. There are two main steps to this backup, doing a dump of the database to begin with, then to copy this to the remote machine, and of course the last task is to mail the administration team with the results.

To dump the mysql database, we could use the mysqldump command, now to totally automate this, you would have to either 1) use no password for the username on the database, or 2) (which I'd prefer) is to put the password in plain text in a shell script, remember the backup directory is 700. Now in mysql I'm assuming you know how to add a user that only has select access to the database, so if anybody did find this password, they couldn't corrupt any data, just read the database as it is. so take we want to just dump one database, called foo, with the username of foobackup and the password of foopass, we'd run;

mysqldump -u foobackup --password="foopassword" foo > foo-database .

Of course this is great the first time, but it doesn't quite scale so well when making a copy, so something like;

mysqldump -u foobackup --password="foopassword" foo > ~/foodb/foo`date +%d%m%y`

which would put the backup in a directory called foodb on the machine, and the filename would look something like foo-011203 (for the first of December 2003).

Once this is done, we need to copy the file, so once again try this manually;

/usr/bin/scp /home/backup/foodb/foo`date +%d%m%y ` 
hostb:~/foodb/

which should just work fine and copy the files to hostb, in the directory ~/foodb/ . So now that you can see it all works on the command line it's time for a basic script, like

#!/bin/sh
echo 'backing up d/b locally'
mysqldump -u foobackup --password="foopassword" foo > ~/foodb/foo`date +%d%m%y`
echo 'copying to hostb'
scp /home/backup/foodb/foo`date +%d%m%y`
hostb:~/foodb/

Save this to a file called, say, foobackup.sh, inside /home/backup/, then execute crontab -e and for example if you want to run this at 1am nightly, you'd put in

0 1 * * * /bin/sh /home/backup/foo.sh

4 - Doing a remote action based on a real time event.

For this example, I'll take something simple like monitoring if apache is alive, and restarting it if it isn't alive. Once again I'm going to operate on the least privilge needed to do the action, which in this case means using it's own user, ssh keys, and sudo . In this case we'll use a python script to just try and connect to port 80 of the webserver, if it isn't up, it'll start the serviec, it's a basic do an action on a remote machine based on a socket, which could be applied in many cases. So first things first, we'll have the enviroment, if you were to use the backup user, executing the script from hosta and the webserver being on hostb, all you have to setup at this stage is sudo on hostb, it's pretty much the same steps as above for setting up the keys.

To allow the backup user execute apachectl, we'll put this into /etc/sudoers;

backup souken = NOPASSWD: /usr/sbin/apachectl

which allows the backup user execute the apachectl command as root, without prompting for a password, note: in this case the machine name is souken, just type host and put that value there instead of souken. After this it's just running the script, the script will try to connect to port 80 on souken, if it doesn't work it'll execute the command;

ssh hostb apachectl restart

it'll use restart instead of start just to be safe instead of some obsecure problem. I have put the basic code for one here (Note: in real life it would log to a better place, and everything would be configurable, but you get the idea).

This article was originally published in the Linux Gazette