[PLUG] strange behavior of "cp" command noticed when copying very large files
Keith Lofstrom
keithl at gate.kl-ic.com
Thu Mar 29 17:43:33 UTC 2012
On Wed, Mar 28, 2012 at 07:08:25PM -0700, website reader wrote:
> Is the odd behavior of the "cp" command which I have observed normal?
>
> I have a large partition (call it /large) set up which is about 3
> terabytes in size.
>
> When I copy a 24 gigabyte file in that partition to another file in
> the same partition, I notice that system memory starts to get used up.
> In fact almost all of the 24.733 gigabytes of memory is eventually
> used, with only 500 meg or less left for system use.
If you are doing the "cp" as root, that takes priority over user
commands. Try lowering the priority with "nice". If the process
is already running, try using "renice". As far as the kernel
knows, moving this big file is very urgent; tell it otherwise.
There is also a command called "ionice", but I haven't used it.
If you are moving files over the network, or you are copying a
big file onto an older-but-nearly-identical file, use the rsync
command with --bwlimit
Linux has lots of tweaks for moving big data and tuning systems.
However, it cannot guess what is important to you, and out of
the box, it is tuned for getting big jobs done fast.
Keith
--
Keith Lofstrom keithl at keithl.com Voice (503)-520-1993
KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon"
Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs
More information about the PLUG
mailing list