BuzzCrawler 1.2.0

buzz crawler

Install-Package BuzzCrawler -Version 1.2.0
dotnet add package BuzzCrawler --version 1.2.0
<PackageReference Include="BuzzCrawler" Version="1.2.0" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add BuzzCrawler --version 1.2.0
The NuGet Team does not provide support for this client. Please contact its maintainers for support.

buzz-crawler

Build status
Build Status
Coverage Status
NuGet Badge

Features

  1. crawling chinese forum page that generated by discuz v2, v3.1, v3.2
  2. crawling korean hungryApp forum page
  3. crawling korean inven forum page
  4. crawling japanese shitaraba forum page
  5. crawling your own forum page

Quick start

var crawler = BuzzCrawlerFactory.GetBBSCrawler(
    BBSType.Discuz_v2,
    new BBSCrawleTarget(
        articleUrl: "http://bbs.site-that-you-want-to-crawle.com/forum.php?mod=viewthread&tid=",
        articleNoQuerystring: "tid",
        listUrl: "http://bbs.site-that-you-want-to-crawle.com/forum.php?mod=forumdisplay&fid=&orderby=dateline&filter=author&page=",
        pageNoQuerystring: "page",
        bbsInstanceKey: "111",
        bbsInstanceKeyQuerystring: "fid"));
crawler.OnCrawleComplete += (crawledCount, lastBuzzWriteDate) => Console.WriteLine($"crawled complete. {crawledCount} buzz crawled. last writedate is {lastBuzzWriteDate}");
crawler.OnNewBuzzCrawled += buzz => Console.WriteLine($"buzz crawled{Environment.NewLine}{JsonConvert.SerializeObject(buzz)}");
crawler.Crawle();

for more information, please watch this video

buzz-crawler

Build status
Build Status
Coverage Status
NuGet Badge

Features

  1. crawling chinese forum page that generated by discuz v2, v3.1, v3.2
  2. crawling korean hungryApp forum page
  3. crawling korean inven forum page
  4. crawling japanese shitaraba forum page
  5. crawling your own forum page

Quick start

var crawler = BuzzCrawlerFactory.GetBBSCrawler(
    BBSType.Discuz_v2,
    new BBSCrawleTarget(
        articleUrl: "http://bbs.site-that-you-want-to-crawle.com/forum.php?mod=viewthread&tid=",
        articleNoQuerystring: "tid",
        listUrl: "http://bbs.site-that-you-want-to-crawle.com/forum.php?mod=forumdisplay&fid=&orderby=dateline&filter=author&page=",
        pageNoQuerystring: "page",
        bbsInstanceKey: "111",
        bbsInstanceKeyQuerystring: "fid"));
crawler.OnCrawleComplete += (crawledCount, lastBuzzWriteDate) => Console.WriteLine($"crawled complete. {crawledCount} buzz crawled. last writedate is {lastBuzzWriteDate}");
crawler.OnNewBuzzCrawled += buzz => Console.WriteLine($"buzz crawled{Environment.NewLine}{JsonConvert.SerializeObject(buzz)}");
crawler.Crawle();

for more information, please watch this video

Release Notes

support gameinven site crawling

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version History

Version Downloads Last updated
1.2.0 416 4/25/2018
1.1.0 437 3/3/2017
1.0.2 423 1/26/2017
1.0.1 547 12/22/2016
1.0.0 426 10/25/2016