[PLUG] Rsyncd And Performance

[Russell Senior]

>>>>> "Wil" == Wil Cooley <wcooley at nakedape.cc> writes:

Russell> IIRC, rsync will do it's full deal if any of the files
Russell> metadata differ: time stamps, size, permissions, etc.  So if
Russell> your data files time stamps are changing, then you don't need
Russell> to use -c.  The -c will force a full checksum for every file,
Russell> so assuming that your rsync includes unchanged (in any way)
Russell> files, dropping the -c will make it go faster.

Wil> I'm not sure about that.  I have the suspicion that if the
Wil> timestamps differ, rsync will go into its block-by-block checksum
Wil> comparison.  OTOH, if you use the -c option, it does a whole-file
Wil> checksum to determine if they files are in fact different, and
Wil> only if they are then it does the block differential algorithm.

Here's what the manpage says:

       -c, --checksum

              This forces the sender to checksum every regular file
              using a 128-bit MD4 checksum.  It does this during the
              initial file-system scan as it builds the list of all
              available files.  The receiver then checksums its
              version of each file (if it exists and it has the same
              size as its sender-side counterpart) in order to decide
              which files need to be updated: files with either a
                                              ^^^^^^^^^^^^^^^^^^^
              changed size or a changed checksum are selected for
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              transfer.  Since this whole-file checksumming of all
              ^^^^^^^^
              files on both sides of the connection occurs in addition
              to the automatic checksum verifications that occur
              during a file's transfer, this option can be quite
              slow. [...]

So it looks like I was recalling incorrectly (thanks Wil, for forcing
me to read the manpage!).  The -c avoids the block differential
algorithm in the cases Jason is worried about (timestamp changed,
contents not changed), so it could be faster.

However, if your rsync included bystander files that had unchanged
size, modification timestamps and content, then -c would force a full
checksum where it wasn't needed, and overall the process could be
expected to be slower.


-- 
Russell Senior, Secretary
russell at personaltelco.net