[PLUG] wget and politeness

[Paul Heinlein]

On Fri, 24 Dec 2004, Keith Lofstrom wrote:

> So, the question;  is there some wget etiquette that I don't know 
> about?

My web site doesn't get enough traffic to make robots a true nuisance, 
but I do occasionally see someone run through the entire site in the 
span of a couple minutes -- which strikes me as annoying.

Have you tried the --wait option to specify a timeout between 
retrievals? E.g.,

   wget --mirror --wait=2m http://blah.foo.com/

The retrieval process is slower, but much kinder to the remote server.

-- Paul Heinlein <heinlein at madboa.com>