Spider a web host with many mechanize agents concurrently, and generate an asset JSON and/or an XML sitemap of the result