Dementia Posted October 5, 2012 Share Posted October 5, 2012 Anyone ever use Wget or something similar to crawl an online journal database? What if I wanted to extract every single .pdf and have them placed neatly into sub-directories, organized by journal name? Link to comment Share on other sites More sharing options...
mcbpete Posted October 5, 2012 Share Posted October 5, 2012 wget -r -np -A.pdf whatever-the-url-is Erm possibly - haven't used it in years, don't blame me if you end up downloading the whole internet ... I seem to remember something about robots.txt preventing crawling (mass downloading) on sites as well Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.