Andrew J. Nelson
18 April 2013
This article will show you how to write a script that allows you to quickly and efficiently update your production webserver from your Linux staging server (or development server, whichever term you prefer). We'll modify the hosts file to make local development easy, securely connect to the production server using SSH, back up the production site before updating, and then export the development website to the production server using RSYNC, all in one single script.
If you use FTP to update your website, I am sure you know the frustration of backing up before the update, or forgetting to include a file that was modified. RSYNC solves those problems, but more on that later.
This how-to assumes that you have two Linux webservers, one used as a staging server and the other as your production platform, and that you are moderately comfortable navigating in a shell environment. If you need to set up your LAMP server, you can refer to WebGnuru's Slackware LAMP How-To. This how-to was defined using Slackware64 14.0, but the principles are the same for any Linux distribution.
For the purposes of this article we will assume the following things. You will need to change these variables to your situation, of course.
When you are working on your development website, you want to be able to browse it just like it is the live site, without having to modify URLs to localhost or anything like that. There are multiple ways to do this, but the simplest is to modify your hosts file to point the domain name to localhost.
The hosts file is loaded before any DNS servers, and takes precedence over them. Thus, when your hosts file has an entry pointing widgets.com to 127.0.0.1, you can use your browser to navigate the development site transparently.
A screenshot of an example hosts_open file is to the right, and the copy commands are below.
cp -v /etc/hosts /home/webdev/hosts.bak
cp -v /etc/hosts /home/webdev/hosts_closed
cp -v /home/webdev/hosts_closed /home/webdev/hosts_open
Our staging server script will overwrite the hosts file with hosts_closed when we are in development mode, and then overwrite it again with hosts_open when we want to export to the production server.
We want the script to be able to automatically SSH from devserv to prodserv without the user, webdev, having to enter a password. In order to do that, we need to establish a trusted relationship between the two hosts with preshared keys. You can read a complete, but brief, WebGnuru how-to on ssh-keygen and sharing, but the highlights are below.
The two steps to the process are to generate the keys on the devserv, then share the public key to prodserv.
ssh-keygen -t rsa
cat .ssh/id_rsa.pub | ssh webdev@prodserv 'cat >> .ssh/authorized_keys'
Now that we have an automated, secure connection from devserv to prodserv, we can go about utizing that SSH tunnel to backup the live website and export the development site to the production server with RSYNC.
RSYNC can be used to completely copy a directory structure from one location to another quickly and securely. The beauty of this utility is that it will compare the source directory to the target directory, and only copy over what is different. If only a portion of a file has changed, it doesn't even copy the whole file -- just those bits that are different. We will utilize this powerful tool to first backup the live website, then update it. You can learn much more about how RSYNC works at this RSYNC incremental backup tutorial.
The backup portion of the script assumes that this is just an "emergency" live backup prior to updating, in case something goes wrong, and is overwritten each time the production site is updated. There absolutely should be a second backup routine that meets your business needs outside of this backup.
RSYNC cannot copy a remote directory to a remote directory, thus the need for an SSH tunnel to execute the live backup. The command is straight forward: rsync [options] [source] [target]. In our example, the backup command would be (all one line):
ssh firstname.lastname@example.org "rsync -avh --delete /var/www/htdocs /backups/websites/live"
The rsync options are to run the process in archive mode, verbosely, in human readable format, and to delete any files in the target directory which have been deleted in the source directory. Once the command is executed, the SSH tunnel is automatically closed.
Now we can export our development version of the website to the production server. RSYNC can copy from a local directory to a remote directory without first opening an SSH tunnel. However, it still utilizes SSH to execute the transfer, thus our data is secure and we still don't need to enter a password.
rsync -avh --delete /var/www/htdocs/ widgets.com:/var/www/htdocs/
A word of caution: if you are not familiar with rsync, or have not read the tutorial linked above, the inclusion or exclusion of trailing slashes in the source directory is important.
So let's create our staging script. We will creatively name it web_updater.sh. Using your favorite text editor, create web_updater.sh in your home directory. The script uses variables to define the rsync options, as that makes maintenance much easier.
# Swap to the open hosts file
cp -v /home/webdev/hosts_open /etc/hosts
# Backup the production website prior to updating it, via rsync and ssh. You
# must ssh to the production server as rsync does not allow both the target and
# source direcories to be remote.
# Source directory of the backup
# Target directory of the backup
# Backup options
# Run the backup
ssh email@example.com "rsync $OPT1 $SRC1 $TRG1"
# Export the staging website to the production server.
# Source directory of the export
# Target directory of the export
# Export options
# Run the update
rsync $OPT2 $SRC2 $TRG2
Return to the closed hosts file
cp -v /home/webdev/hosts_closed /etc/hosts
Don't fortget to chmod the file to make it executable, and the next time you need to update the live website, simply execute it.
Well, our script now allows us to control whether we browse the production or development website by modifying our hosts file, backup the live website before updating, and update the site without pesky FTP transfers. I hope this helps you more efficiently and securely maintain your corner of the web.
If you have any questions, concerns, suggestions, or constructive criticism, please email me.