Staging Server Script

Andrew J. Nelson
18 April 2013

This article will show you how to write a script that allows you to quickly and efficiently update your production webserver from your Linux staging server (or development server, whichever term you prefer). We'll modify the hosts file to make local development easy, securely connect to the production server using SSH, back up the production site before updating, and then export the development website to the production server using RSYNC, all in one single script.

If you use FTP to update your website, I am sure you know the frustration of backing up before the update, or forgetting to include a file that was modified. RSYNC solves those problems, but more on that later.

Required Materials and Prior Knowledge

This how-to assumes that you have two Linux webservers, one used as a staging server and the other as your production platform, and that you are moderately comfortable navigating in a shell environment. If you need to set up your LAMP server, you can refer to WebGnuru's Slackware LAMP How-To. This how-to was defined using Slackware64 14.0, but the principles are the same for any Linux distribution.

For the purposes of this article we will assume the following things. You will need to change these variables to your situation, of course.

  • A staging server with the hostname devserv.
  • A production server with the hostname prodserv.
  • An account on both hosts with the same username webdev.
  • That prodserv has a backup depository /backups/website/
  • A domain name for the website of widgets.com.

Step 1: The Hosts File

A screenshot of hosts_closed.

A hosts_closed example. Click to enlarge.

When you are working on your development website, you want to be able to browse it just like it is the live site, without having to modify URLs to localhost or anything like that. There are multiple ways to do this, but the simplest is to modify your hosts file to point the domain name to localhost.

The hosts file is loaded before any DNS servers, and takes precedence over them. Thus, when your hosts file has an entry pointing widgets.com to 127.0.0.1, you can use your browser to navigate the development site transparently.

  1. On devserv, backup your hosts file, which lives at /etc/hosts to your home directory.
  2. Make another copy of hosts to your home directory with a new name of hosts_closed.
  3. Edit hosts_closed so that widget.com points to your localhost IP address.
  4. Make a copy of hosts_closed named hosts_open.
  5. Edit hosts_open so that there is # in front of the widgets.com entry, remarking it out.

A screenshot of an example hosts_open file is to the right, and the copy commands are below.

  • cp -v /etc/hosts /home/webdev/hosts.bak
  • cp -v /etc/hosts /home/webdev/hosts_closed
  • cp -v /home/webdev/hosts_closed /home/webdev/hosts_open

Our staging server script will overwrite the hosts file with hosts_closed when we are in development mode, and then overwrite it again with hosts_open when we want to export to the production server.

Part 2: Securely Connecting With SSH

We want the script to be able to automatically SSH from devserv to prodserv without the user, webdev, having to enter a password. In order to do that, we need to establish a trusted relationship between the two hosts with preshared keys. You can read a complete, but brief, WebGnuru how-to on ssh-keygen and sharing, but the highlights are below.

The two steps to the process are to generate the keys on the devserv, then share the public key to prodserv.

  • ssh-keygen -t rsa
  • cat .ssh/id_rsa.pub | ssh webdev@prodserv 'cat >> .ssh/authorized_keys'

Now that we have an automated, secure connection from devserv to prodserv, we can go about utizing that SSH tunnel to backup the live website and export the development site to the production server with RSYNC.

Part 3: Backup And Export With RSYNC

RSYNC can be used to completely copy a directory structure from one location to another quickly and securely. The beauty of this utility is that it will compare the source directory to the target directory, and only copy over what is different. If only a portion of a file has changed, it doesn't even copy the whole file -- just those bits that are different. We will utilize this powerful tool to first backup the live website, then update it. You can learn much more about how RSYNC works at this RSYNC incremental backup tutorial.

The backup portion of the script assumes that this is just an "emergency" live backup prior to updating, in case something goes wrong, and is overwritten each time the production site is updated. There absolutely should be a second backup routine that meets your business needs outside of this backup.

RSYNC cannot copy a remote directory to a remote directory, thus the need for an SSH tunnel to execute the live backup. The command is straight forward: rsync [options] [source] [target]. In our example, the backup command would be (all one line):

  • ssh webdev@widgets.com "rsync -avh --delete /var/www/htdocs /backups/websites/live"

The rsync options are to run the process in archive mode, verbosely, in human readable format, and to delete any files in the target directory which have been deleted in the source directory. Once the command is executed, the SSH tunnel is automatically closed.

Now we can export our development version of the website to the production server. RSYNC can copy from a local directory to a remote directory without first opening an SSH tunnel. However, it still utilizes SSH to execute the transfer, thus our data is secure and we still don't need to enter a password.

  • rsync -avh --delete /var/www/htdocs/ widgets.com:/var/www/htdocs/

A word of caution: if you are not familiar with rsync, or have not read the tutorial linked above, the inclusion or exclusion of trailing slashes in the source directory is important.

Step 4: Putting It All Together In A Script

So let's create our staging script. We will creatively name it web_updater.sh. Using your favorite text editor, create web_updater.sh in your home directory. The script uses variables to define the rsync options, as that makes maintenance much easier.

  • #!/bin/bash
  •  
  • # Swap to the open hosts file
  • cp -v /home/webdev/hosts_open /etc/hosts
  •  
  • # Backup the production website prior to updating it, via rsync and ssh. You
  • # must ssh to the production server as rsync does not allow both the target and
  • # source direcories to be remote.
  •  
  • # Source directory of the backup
  • SRC1="/var/www/htdocs"
  •  
  • # Target directory of the backup
  • TRG1="/backups/website"
  •  
  • # Backup options
  • OPT1="-avh --delete"
  •  
  • # Run the backup
  • ssh webdev@widgets.com "rsync $OPT1 $SRC1 $TRG1"
  •  
  • # Export the staging website to the production server.
  •  
  • # Source directory of the export
  • SRC2="/var/www/htdocs/"
  •  
  • # Target directory of the export
  • TRG2="widgets.com:/var/www/htdocs"
  •  
  • # Export options
  • OPT2="-avh --delete"
  •  
  • # Run the update
  • rsync $OPT2 $SRC2 $TRG2
  •  
  • Return to the closed hosts file
  • cp -v /home/webdev/hosts_closed /etc/hosts
  •  
  • # EOF

Don't fortget to chmod the file to make it executable, and the next time you need to update the live website, simply execute it.

Conclusion

The tux logo for WG.

Share this article on Google+ or your favorite social media. Thanks!

Well, our script now allows us to control whether we browse the production or development website by modifying our hosts file, backup the live website before updating, and update the site without pesky FTP transfers. I hope this helps you more efficiently and securely maintain your corner of the web.

If you have any questions, concerns, suggestions, or constructive criticism, please email me.

Copyright © 2008 - 2013 Andrew Nelson under the GPL v3 License except where noted.
Please see the README file for full licensing disclosure and credits.