[PLUG] strange behavior of "cp" command noticed when copying very large files

Keith Lofstrom keithl at gate.kl-ic.com
Thu Mar 29 17:43:33 UTC 2012


On Wed, Mar 28, 2012 at 07:08:25PM -0700, website reader wrote:
> Is the odd behavior of the "cp" command which I have observed normal?
> 
> I have a large partition (call it /large) set up which is about 3
> terabytes in size.
> 
> When I copy a 24 gigabyte file in that partition to another file in
> the same partition, I notice that system memory starts to get used up.
>  In fact almost all of the 24.733 gigabytes of memory is eventually
> used, with only 500 meg or less left for system use.

If you are doing the "cp" as root, that takes priority over user
commands.  Try lowering the priority with "nice".  If the process
is already running, try using "renice".  As far as the kernel
knows, moving this big file is very urgent;  tell it otherwise.

There is also a command called "ionice", but I haven't used it. 
If you are moving files over the network, or you are copying a
big file onto an older-but-nearly-identical file, use the rsync
command with --bwlimit

Linux has lots of tweaks for moving big data and tuning systems.
However, it cannot guess what is important to you, and out of 
the box, it is tuned for getting big jobs done fast.


Keith
-- 
Keith Lofstrom          keithl at keithl.com         Voice (503)-520-1993
KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon"
Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs



More information about the PLUG mailing list