• Abot is an open source C# web crawler built for speed and flexibility. It takes care of the low level plumbing (multithreading, http requests, scheduling, link parsing, etc..). You just register for events to... More information
    • 214,139 total downloads
    • last updated 1/19/2016
    • Latest version: 1.0.8
    • utilities common
    A robots.txt parser written in c#. This is an unofficial fork of NRobots.txt on codeplex with extended functionality. For full documentation see
  • A powerful C# web crawler that makes advanced crawling features easy to use. AbotX builds upon the open source Abot C# Web Crawler by providing a powerful set of wrappers and extensions.
    • 14,817 total downloads
    • last updated 9/27/2017
    • Latest version: 1.0.9
    Search engines and social networks are always trying to crawl your pages, but they only see the javascript tags... We render your javascript in a browser, save the static HTML, and you return that to the... More information
    • 14,607 total downloads
    • last updated 1/15/2018
    • Latest version: 1.0.15
    • utilities common
    C# common libs, utilities and helpers.