A Web spidering library: handles robots.txt, scraping, finding more links, and doing it all over again.
Required Ruby Version
>= 0
Authors
John Nagro
Versions
- 0.8.0 July 06, 2026 (10 KB)
- 0.7.0 September 08, 2025 (10 KB)
- 0.6.0 September 08, 2025 (10 KB)
- 0.5.4 June 04, 2020 (15.5 KB)
- 0.5.3 April 23, 2018 (15 KB)
- 0.5.1 September 04, 2016 (15 KB)