[PLUG] new wget question (robots.txt)

john morgali aarghj at yahoo.com
Tue Dec 28 20:58:42 UTC 2004


I have looked and dont seem to be seeing if there is a
way to make wget ignore the robots.txt file and grab a
file or directory from a site.  I was just thinking of
this because of the previous thread on wget.  Anyway,
let me know and thanks in advance. 




=====
Pursuant to U.S. code, title 47, Chapter 5, Subchapter II, Section 227, and consistent with Oregon State Law, any and all unsolicited  commercial E-mail sent to this address is subject to a consulting fee.U.S. E-Mailing denotes acceptance of these terms.Consult http://www.law.cornell.edu/uscode/47/227.html for details.

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 



More information about the PLUG mailing list