cobweb 1.0.4

Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more performant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.

Gemfile:
=

install:
=

Versions:

1.2.1 January 09, 2021 (9.28 MB)
1.2.0 February 25, 2019 (9.28 MB)
1.1.0 November 06, 2015 (9.28 MB)
1.0.29 November 05, 2015 (9.28 MB)
1.0.28 March 20, 2015 (9.28 MB)
1.0.4 January 15, 2013 (9.28 MB)

Show all versions (94 total)

Runtime Dependencies (11):

addressable >= 0

awesome_print >= 0

haml >= 0

json >= 0

namespaced_redis >= 0

nokogiri >= 0

redis >= 0

resque >= 0

rspec >= 0

sinatra >= 0

thin >= 0

Show all transitive dependencies

Owners:

Authors:

Stewart McKee

cobweb 1.0.4

Gemfile:
=

install:
=

Versions:

Runtime Dependencies (11):

Owners:

Authors:

SHA 256 checksum:

Total downloads 354,853

For this version 3,819

Version Released:
January 15, 2013 8:10pm

Licenses:
N/A

Required Ruby Version: None

Links:

cobweb 1.0.4

Gemfile: =

install: =

Versions:

Runtime Dependencies (11):

Owners:

Authors:

SHA 256 checksum:

Total downloads 354,853

For this version 3,819

Version Released: January 15, 2013 8:10pm

Licenses: N/A

Required Ruby Version: None

Links:

Gemfile:
=

install:
=

Version Released:
January 15, 2013 8:10pm

Licenses:
N/A