A robots.txt parser written in c#. This is an unofficial fork of NRobots.txt on codeplex with extended functionality. For full documentation see https://nrobots.codeplex.com/.
Install-Package NRobotsPatched -Version 1.0.8
dotnet add package NRobotsPatched --version 1.0.8
<PackageReference Include="NRobotsPatched" Version="1.0.8" />
paket add NRobotsPatched --version 1.0.8
This package has no dependencies.
Showing the top 1 GitHub repositories that depend on NRobotsPatched:
C# web crawler built for speed and flexibility. Please star this project! +1. Contact me with exciting opportunities!!