[PLUG] copying large numbers of files

[Ian Burrell]

On 3/30/06, Brent Rieck <bsr at spek.org> wrote:
> Russell Senior wrote:
> > Out of curiosity, which file system(s) are these going from/to?
>
> >From a NFS mounted ext3 partition  to a local reiserfs partition - the
> network appears to be saturated too.  But yay! it just finished copying.
>

You might want to see if you can get on the file server and transfer
directly from there.  My guess is that the slowness is caused by
latency in requesting each file.  I am pretty sure NFS requires a
round-trip message for each file; this adds up for a million files. 
Reading lots of small files is faster on a local disk because Linux
does caching of file metadata and blocks.

 - Ian