[PLUG] Faster giant disk copy

Russell Senior seniorr at aracnet.com
Sat Aug 14 14:45:03 UTC 2004


>>>>> "Russell" == Russell Senior <seniorr at aracnet.com> writes:

Russell> I don't know if it is better or even faster, but you might
Russell> try repeating the snapshots in the order they were initially
Russell> done:

FWIW, I am doing this right now in order to consolidate snapshots onto
larger media.  I am doing so from two separate archive disks mounted
at /mnt2 and /mnt3 and writing the results to a new disk mounted at
/mnt.  Each of my snapshots are date-stamped in ISO format
YYYY-MM-DD-HHMM.

First I create a list of the snapshots from the two source media and
then sort them chronologically:

  # find /mnt2/myhost -maxdepth 1 -mindepth 1 > /tmp/myhost-snapshots
  # find /mnt3/myhost -maxdepth 1 -mindepth 1 >> /tmp/myhost-snapshots
  # sort -t'/' -k 3 /tmp/myhost-snapshots > /tmp/myhost-snapshots-chron

Then use awk to generate an "almost correct" script, lagging the
snapshot link-dest directory variable:

  # awk -F/ '{ printf("time rsync -v -a -H --delete --numeric-ids --link-dest=../%s/ %s/ %s/\n",last,$0,$4); last = $4 }' /tmp/myhost-snapshots-chron > /tmp/recreate-snapshot-history-myhost.sh

Then I modified the first line to remove the link-dest parameter, and
starting from an appropriate current working directory (the new
destination of the snapshots), I run:

  # sh /tmp/recreate-snapshot-history-myhost.sh

and away it goes, recreating the snapshots in order, creating
hardlinks to the locally most recent previous snapshot. 

I don't know what else Dirvish does, but that recreates the basic
snapshot data, and in a timely fashion.


-- 
Russell Senior         ``I have nine fingers; you have ten.''
seniorr at aracnet.com




More information about the PLUG mailing list