[PLUG] When mounted doesn't mean mounted

Michael M. Moore michael at writemoore.net
Thu Jun 25 21:23:12 UTC 2009


On Thu, 2009-06-25 at 11:05 -0700, John Jason Jordan wrote:
> 
> OK, I created the globbing file. At 9:32 am I started it. There was
> a /media/disk2/Full_system_backup folder already in existence, created
> when I ran rdiff-backup the first time a month ago using pybackpack.
> The lights on both disks are flickering, and Nautilus reports the date
> on the destination folder has changed to today, and I see no new
> folders. I think I have success. <crossing fingers>
> 
> However, I got an error on the .gvfs file:
> 
> jjj at Devil7:~$ sudo rdiff-backup
> --include-globbing-filelist /home/jjj/rdiff_excludes.txt / /media/disk2/Full_system_backup
> ListError home/jjj/.gtkrc-1.2-gnome2/.gvfs [Errno 13] Permission
> denied: '/home/jjj/.gvfs'
> 
> I assume that means it was not backed up. I wanted to exclude it
> anyway, so fine. But I don't like error messages.

If you don't want the error message, put it in your file list:

- /home/jjj/.gtkrc-1.2-gnome2/.gvfs
- /home/jjj/.gvfs

(I have no ~/.gkkrc-1.2-gnome2 directory.  In fact, I have no ~/.gtk
anything directory.)

> I also got pages and pages of additional errors. They were all of three
> types, examples here:
> 
> UpdateError
> sys/devices/pci0000:00/0000:00:1d.7/usb2/usb_endpoint/usbdev2.1_ep00/bmAttributes
> Updated mirror temp
> file /media/disk2/Full_system_backup/sys/devices/pci0000:00/0000:00:1d.7/usb2/usb_endpoint/usbdev2.1_ep00/rdiff-backup.tmp.12219
> does not match source

Don't /sys/devices and other data under /sys change with each boot, or
each time a device is mounted?  I thought one of the motivations for
using UUIDs in /etc/fstab instead of the more traditional /dev/hda1 (or
2 or 3, etc) was that /dev entries are now dynamically assigned.  Sysfs
is always going to be somewhat different between boots or mounts, as I
understand it anyway.  I guess that begs the question, what is the value
in backing up /sys?

> 
> And
> 
> Exception '[Errno 19] No such device' raised of class '<type
> 'exceptions.IOError'>': File
> "/var/lib/python-support/python2.6/rdiff_backup/robust.py", line 32, in
> check_common_error try: return function(*args)
> 
> And 
> 
> Traceback (most recent call last):
>   File "/usr/bin/rdiff-backup", line 30, in <module>
>     rdiff_backup.Main.error_check_Main(sys.argv[1:])
> 
> Can someone translate these to English for me?

Not I ... hopefully somebody else can.  I wonder if they resulted from
the fact that you were incrementing a backup that had previously been
created using pybackpack.  I don't know what kind of issues that can
introduce -- it's possible that pybackpack adds some elements that
aren't present when you're using rdiff-backup by itself.  I guess the
thing to do would be run the job again and see if you still get the same
errors.  If you don't, then it was just temporary dissonance caused by
the differences in the methods you used.  If you do, it might be safer
to delete the whole backup and do it again using rdiff-backup and only
rdiff-backup.  Generally speaking, mixing and matching different
programs to do the same job is probably not a good idea.  (I know
pybackpack is supposedly just a front-end for rdiff-backup, but
still ... it possibly introduces some subtle changes in the way
rdiff-backup works.)

> > Eventually you will most likely want to delete some of the older
> > increments.  You do that by running:
> > 
> > rdiff-backup --remove-older-than 1W /media/disk2/Full_system_backup
> > 
> > (That will remove increments older than one week.  You can remove older
> > than one year, two months, three days, four hours, whatever.  Adjust the
> > "1W" as you wish.)
> 
> I have written textbooks where the files never change, but need to be
> saved for maybe ten years, for when I need to do a new edition. Every
> six months or so I want to start a new Full_system_backup folder and
> save the old one forever as a snapshot of everything as of the date it
> was last incremented. So I guess the way to do that would be to do an
> incremental, then a remove-older-than command for everything older than
> a few hours, rename the old folder by (e.g.) appending the (then) date,
> and create a new Full_system_backup folder for the next backup.

I guess.  That means you'll be saving full system backups for 10 years
or more?  That seems like, well, forever, in computer time.  It's hard
to imagine what Ubuntu will be like in 10 years, or whether it will
still be around.  You'll be storing an awful lot of irrelevant, useless
data just for the sake of holding onto a few files that won't change
anyway.  You could burn those files to CD/DVD, checking them every six
months to make sure they're still readable; you could store them in a
GMail account (if they don't add up to more than 7GB) or possibly Google
Documents; you could store them remotely elsewhere with any number of
services that exist for that purpose (Dropbox, Amazon A3, soon Ubuntu
One, etc.).  You could do all of these things since the files will be
identical & unchanged, and be relatively sure that at least one of your
methods will not fail over the next decade.  But yes, if you do an
incremental backup, then remove all the older increments, you will be
left with only your most recent backup.

> 
> Is there a way to get a progress bar, or "percentage completed" that
> increments, or maybe a "time remaining" number while it is running?
> Also, at the end it would be nice to have statistics written to a file
> in ~/, or at least displayed in the terminal.

rdiff-backup does write a statistics file by default, though that can be
disabled with the --no-file-statistics option.  If you want to see the
statistics at the conclusion of the job, use the --print-statistics
option.  Otherwise (from the man page):

        STATISTICS
        Every session rdiff-backup saves various statistics into two
        files, the session statistics file at
        
        	rdiff-backup-data/session_statistics.<time>.data
        
        and the directory statistics file at
        
        	rdiff-backup-data/directory_statistics.<time>.data
        
        They are both text files and contain similar information: how
        many files changed, how many were deleted, the total size of
        increment files created, etc.  However, the session statistics
        file is intended to be very readable and only describes the
        session as a whole.  The  directory statistics file is more
        compact (and slightly less readable) but describes every
        directory backed up.  It also may be compressed to save space.
        
        Statistics-related options include --print-statistics and
        --null-separator.
        
These files will be in your backup directory.  There's also a log file
for the whole job.

I'm not sure if rdiff-backup can display a progress bar.  If you use the
--terminal-verbosity [0-9] option, it will display more or less
information in the terminal about what it is doing.  The default is
--terminal-verbosity 3, I believe.  You could try cranking it up to
--terminal-verbosity 9 and see if that gets you a progress bar.  I have
used --terminal-verbosity 5 when I've run rdiff-backup manually.  It
doesn't show a progress bar, but it does print out the names of all the
files it is backing up and I can get a good idea of how far along it is
from that.  It transverses the file system in standard order, the order
that would be displayed if you ran

	$ls /

I don't really think a proper progress bar is possible since it has no
way of knowing the size of the files are that it has yet to hit.  If are
backing up 10 1MB text files and 10 700MB video files, the latter will
take a lot longer to backup than the former, so it would be misleading
to say you're 50% done when one or the other group of files is
backed-up.

-- 
Michael M.




More information about the PLUG mailing list