83 packages returned for Tags:"crawler"

A .NET Standard web crawling library similar to WebMagic and Scrapy. It is a lightweight ,efficient and fast high-level web crawling & scraping framework for .NET
Data Extracting SDK
Library allows you to extract different data from text or web sources, analyze text content, extract different information and write your own data mining & extracting applications and services.
It helps you to use HAP in easier and meaningful way via Reflection. It works somehow like Entity-Framework. Go to wiki in github page for tutorial : https://github.com/parsalotfy/HtmlAgilityPack_Helper/wiki
Crawl and index your (static) asp.net website for searching using an MSBuild target. Simple search query interface. Uses Lucene.Net and NCrawler.
  • 868 total downloads
  • last updated 11/21/2012
  • Latest version: 1.0.0
  • Crawler Web
A minimal multithreaded web crawler