SUMMARY: RSYNC or RDIST

From: m \ (boris@medicalogic.com)
Date: Fri Sep 03 1999 - 09:33:21 CDT


QUESTION: as a temporary method of distributing web site files to various locations, we are considering using either RSYNC or RDIST. does anyone have any positive or negative experience or warnings using either on sun systems?

ANSWER: almost universal consensus that RSYNC was a better product. Gustavo Chavez found one bug in RSYNC and has reported it to Andrew Tridgell (at www.samba.org ). also mentioned: SRFS from Veritas and OpenDeploy from Interwoven. here are some snippets:

a. Patrick Shannon pshannon@macromedia.com - faster than RDIST, use it through SSH.

b. Dan Stromberg strombrg@nis.acs.uci.edu - I've never cared for RDIST, which is kind of clumsy and has been a source of many security holes. RSYNC on the other hand, is more efficient, and capable of doing its thing over SSH, a definite plus.

c. Jeff Putsch putsch@unitrode.com Having used both and currently using RSYNC, I feel RSYNC has the following advantages over RDIST: faster application, faster over slow (WAN) links, and pull model for distribution is more stable.

d. Gary Franczyk franczyk@lucent.com RDIST works well for most things.... but, if the site is large, you may as well do it right and use RSYNC... RSYNC is used by some VERY large web sites.

e. Gustavo Chavez gustavo@cpqd.com.br I use RDIST extensively but I intend to use RSYNC instead. The two features of RSYNC that I think are most important is that you can "pull" the synchronization as well as "push" it. RDIST can only "push" things, which forces you to use ~/.rhosts files, compromising your security. Moreover, if you intend to synchronize hundreds of machines, the "push" model simply don't scale.

The other feature is the possibility to create an anonymous RSYNC server. Together, these features enable you to create a really secure synchronization scheme.

There is one catch up though. I use RSYNC for one thing for almost two years. I run it as root everyday in every machine to synchronize a directory under /usr. In two occasions, this procedure corrupted my file systems. The last time it occurred I investigated and found out a bug in RSYNC. The problem occurs when RSYNC discovers that a directory in the mirror was changed into a symbolic link in the master. Instead of removing the client's directory recursively and creating the link, it calls unlink(2) for the directory and creates the symbolic link. If it's running as root, the unlink succeeds leaving the file system corrupted. Then, when the machines reboot, they all stop waiting for a manual FSCK.

I've submitted a bug report a month ago and Andrew Tridgell (RSYNC's maintainer) responded that he would take a look at the problem in about one month. I haven't received any knows since, but I hope it will be fixed soon. After that, I'll try to use RSYNC for everything I use RDIST now.

f. Philip Plane philip@ra.tepapa.govt.nz I use RSYNC for this purpose. Big advantage is using RSYNC with SSH. I only allow SSH and http connections from the office network to the web server. Running RSYNC over SSH means I can do updates without allowing another protocol through the firewall. Clean and easy.

g. Jay Lessert jayl@latticesemi.com I'm currently using RSYNC to mirror IC layout/schematic libraries and CAD tools at three WAN-connected sites. About 500 MB worth. Currently running 2.3.1 and have had zero problems. Simplicity of setup, user-selectable transport mechanism, and built-in compression were the selling features for me compared to RDIST and mirror.

h. eric eric@outlook.net use RSYNC as it supports SSH.

i. Erin Jones erin@younetwork.com we used RDIST. worked fine. one worry is that you are setting host.equiv and opening a security door to your site

j. Jeff Howard Jeff.Howard@agso.gov.au I have never used "RDIST" but I have heard negative things about its security. The following should be read in that context. We use "RSYNC" to maintain our web content every evening. Two of its strengths are:
* its ability to pass only what is necessary across the network to update the remote site. [speed]
* its ability to establish the connection using a secure protocol. In our case we use the secure shell, SSH, to make the connection.

k. bill walker bill.walker@db.com We use RSYNC extensively without any (known) problems. We found it a lot more
efficient than RDIST.

Here's a script we use to call RSYNC :

#!/bin/ksh
################################################################################
#
# Function: Use rsync to copy filesystems between hosts
#
################################################################################
################################################################################
# Setup global variables
# Rsync options ...
# a archive mode, maintains ownerships, permissions, mod and access times.
# v verbose - turn off if log output TOO verbose :)
# n (optional) dry run, what *would* be transferred, nothing actually copied.
# u update - don't replace NEWER files on the target.
# q quiet - useful when run from cron
# t times - keeps mod/access times
# --stats show transfer statistics on completion of command
# --rsync-path where is the rsync binary on the target machine ?
################################################################################
PROG=`basename $0`
PID=$$
TLOG=/tmp/sspsync.$$
LOG=/var/adm/ssp_sync.log
R_BASE="/usr/local/bin"
R_EXEC="${R_BASE}/rsync"
R_OPTS="-a --rsync-path=${R_EXEC} --delete"
WHOTOMAIL="joe.bloggs@xyz.com"

DIRS="/etc /usr/local/whathaveyou"
FILE=""

################################################################################
# Define functions
# Standard date/PID prefix for log messages. Takes $1 as text message.
################################################################################
Message()
{
D=`date +%d/%m/%Y\|%H:%M`
echo "${D}|PID#${PID}|$1" >> ${TLOG}
}

################################################################################
# Must be root to ensure permission to copy all files. Uses OLD version of id.
################################################################################
Check_root()
{
ID=`/usr/xpg4/bin/id -u`
if [ ${ID} != "0" ]
then
   Message "Must be root user to run this program, aborting ..."
   Exit 1
fi
}

################################################################################
# Mail each address in WHOTOMAIL with copy of temp log. Insert $1 as OK/FAIL.
################################################################################
Mail()
{
mesg=$1
D=`date +%d/%m/%Y:%H:%M`

for addr in ${WHOTOMAIL}
do
    cat ${TLOG} | mailx -s "${D} ${PROG}:${MASTER}->${CLIENT} ${mesg}" ${addr}
done

cat ${TLOG} >> ${LOG}
rm ${TLOG}
}

################################################################################
# Copy directory trees, then files. Check for existence before trying copy.
################################################################################
Do_copy()
{
Message "----------------------------------------------------"
for filetype in directories files
do
   if [ ${filetype} = "directories" ]
   then
      for eachdir in ${DIRS}
      do
         if [ -d ${eachdir} ]
         then
            Message "Copying tree \"${eachdir}\" ..."
            $R_EXEC $R_OPTS ${eachdir}/ ${CLIENT}:${eachdir} 2>&1 >> ${TLOG}
            ERR=$?
            if [ ${ERR} != "0" ]
            then
               Message "Error ${ERR} returned copying \"${eachdir}\""
               Exit ${ERR}
            fi
         else
            Message "Directory \"${eachdir}\" not found, continuing ..."
         fi
      done
   else
      for eachfile in ${FILE}
      do
         if [ -f ${eachfile} ]
         then
            Message "Copying file \"${eachfile}\" ..."
            $R_EXEC $R_OPTS ${eachfile} ${CLIENT}:${eachfile} 2>&1 >> ${TLOG}
            ERR=$?
            if [ ${ERR} != "0" ]
            then
               Message "Error ${ERR} returned copying \"${eachfile}\""
               Exit ${ERR}
            fi
         else
            Message "File \"${eachfile}\" not found, continuing ..."
         fi
      done
   fi
done
}

################################################################################
# Parse variable $1 passed from other functions. 0=success, other=failure.
################################################################################
Exit()
{
case $1 in
   0) Message "Completed successfully"
       Mail OK
        exit 0
        ;;
   *) Message "Program exiting with return code $1"
        Mail FAIL
        exit $1
        ;;
esac
}

################################################################################
# Main
# Make sure we have 2 args, 1st is source_hostname, 2nd is target_hostname
################################################################################
case $# in
   2) MASTER=$1
      CLIENT=$2
      ;;
   *) Message "Usage: ${PROG} source_hostname target_hostname"
      Exit 1
      ;;
esac

Check_root
Do_copy
Exit 0

################################################################################
# The end
###############################################################################



This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:13:25 CDT