[PLUG] Bulk Downloads With wget

[wes]

>
>   I've read the wget man page and tried some example syntax without the
> results I want. All I get is the Atlanta response "404 Not Found". Examples
> of what I've tried:
>

what is "the Atlanta response" ? Is that a reference to something I'm not
familiar with?


> [rshepard at salmo ~]$ wget --no-glob
> http://slackware.osuosl.org/slackware-12.0/patches/packages/*.tgz
> --06:33:43--<http://slackware.osuosl.org/slackware-12.0/patches/packages/*.tgz--06:33:43-->
> http://slackware.osuosl.org/slackware-12.0/patches/packages/*.tgz
>            => `*.tgz'
> Resolving slackware.osuosl.org... 140.211.166.134
> Connecting to slackware.osuosl.org|140.211.166.134|:80... connected.
> HTTP request sent, awaiting response... 404 Not Found
> 06:33:44 ERROR 404: Not Found.
>

Unfortunately, wget only supports the use of wildcards over ftp, not http. I
believe OSU's OSL does make things available via ftp, so you should be able
to do that.

ftp://slackware.osuosl.org/pub/slackware/slackware-12.0/patches/packages/

[rshepard at salmo ~]$ wget -A.tgz
> http://slackware.osuosl.org/slackare-12.0/patches/packages/
> --06:38:03--<http://slackware.osuosl.org/slackare-12.0/patches/packages/--06:38:03-->
> http://slackware.osuosl.org/slackare-12.0/patches/packages/
>            =>  `index.html'
> Resolving slackware.osuosl.org... 140.211.166.134
> Connecting to slackware.osuosl.org|140.211.166.134|:80... connected.
> HTTP request sent, awaiting response... 404 Not Found
> 06:38:03 ERROR 404: Not Found.
>

I don't know why this doesn't work, but it doesn't work for me either. I've
never gotten multiple-file fetching to work over http. I have always found
some other workaround first, like using ftp, or creating lists of what I
want to get, and getting them 1 at a time via a shell loop. This is a bit
more advanced though, so I would suggest you take the easy ftp out while you
have the chance :)


>   What syntax should I use? Or, is there a better tool than wget for this
> task?
>

Some may advocate curl, but I am not particularly familiar with that tool.
It may be worth a glance at the man page to see if it will do what you want.
I'm sure it might come in handy some day either way.


> Rich
>

-wes