Skip To Content
Toggle navigation
Packages
Upload
Statistics
Documentation
Downloads
Blog
Sign in
Advanced search filters
Frameworks
Include compatible frameworks
Framework Filter Mode
ALL
ANY
.NET
net8.0
net7.0
net6.0
net5.0
.NET Core
netcoreapp3.1
netcoreapp3.0
netcoreapp2.2
netcoreapp2.1
netcoreapp2.0
netcoreapp1.1
netcoreapp1.0
.NET Standard
netstandard2.1
netstandard2.0
netstandard1.6
netstandard1.5
netstandard1.4
netstandard1.3
netstandard1.2
netstandard1.1
netstandard1.0
.NET Framework
net481
net48
net472
net471
net47
net462
net461
net46
net452
net451
net45
net40
net35
net30
net20
Package type
All types
Dependency
.NET tool
Template
Options
Include prerelease
23 packages returned for Tags:"Robots.
txt"
Sort by
Relevance
Downloads
Recently updated
AspNetCore.
SEOHelper
by:
esty_c
.NET 5.0
.NET Core 3.1
272,919 total downloads
last updated
5/11/2020
Latest version:
1.0.1
Robots.txt
sitemap.xml
SEO
Search Engine management tools. This library helps developers to create routing for robots.txt and sitemap.xml. Also provides facility for creating sitemap.xml file and load existing sitemap.xml file.
RobotsTxt
by:
tsllbn
.NET Framework
52,458 total downloads
last updated
2/19/2014
Latest version:
2014.2.19
Robots.Txt
Parser
Parsing
Library
A robots.txt parser for .NET Supports ; - Allow directives. - Crawl-delay directives. - Sitemap declarations. - * and $ wildcards. See https://bitbucket.org/cagdas/robotstxt for usage examples.
Our.
Umbraco.
RobotsTxtEditor
by:
leekelleher
lottepitcher
MikeMasey
.NET Framework 4.5
32,588 total downloads
last updated
10/27/2019
Latest version:
8.0.0
umbraco
robots.txt
editor
An Umbraco package to edit your robots.txt file
SimpleSitemap
by:
pure krome
.NET 5.0
.NET Core 1.0
.NET Standard 1.2
.NET Framework 4.5.1
77,329 total downloads
last updated
9/24/2020
Latest version:
3.2.0
simple
sitemap
sitemaps
robots
robots.txt
unicorn
magicalunicorn
magical-unicorn
worlddomination
worldomination
SimpleSitemap is a lite library that helps you create web sitemaps for collections or lists of items. These sitemaps follow the Sitemap Protocol. Both sitemapindex and urlset links are generated, based upon...
More information
GenHTTP.
Modules.
Robots
by:
Kaliumhexacyanoferrat
.NET 6.0
17,393 total downloads
last updated
4/23/2024
Latest version:
8.3.0
HTTP
Webserver
C#
Module
Robots
Bots
Robots.txt
Allows to serve a robots instruction file.
RobotsTxt.
Checker
by:
Dobryak
8,967 total downloads
last updated
2/18/2020
Latest version:
0.0.13
utility
robots.txt
check
Small utility, used to check robots.txt on web app and make sure that it is valid. RobotsTxt check is based on https://www.nuget.org/packages/RobotsTxt, but this project is no longer maintained, so I copied it...
More information
RobotRules
by:
vodevil
.NET Framework 4.5
6,591 total downloads
last updated
6/10/2014
Latest version:
1.5.2.4
Robots.txt
robot
crawler
web
A Simple and Powerful Library to Deal with Web Robots Control Strategy.
SiteMaps.
NET
by:
LiveOrDevTrying
.NET 5.0
.NET Core 2.0
17,927 total downloads
last updated
9/25/2020
Latest version:
2.1.3
sitemap
sitemap.xml
sitemaps
robots
robot
robots.txt
SEO
search
engine
optimization
A simple middleware built on reflection designed to support your Search Engine Optimized (SEO) app by dynamically creating a Sitemap.xml and Robots.txt. This package is designed to be very simple for simple use...
More information
TamTam.
ConditionalRobots
by:
tamtam
2,748 total downloads
last updated
4/4/2015
Latest version:
1.0.1.5
robots.txt
allow
disallow
- It adds a robots.txt and a robots_closed.txt to the root of the website - It adds a rewrite rule to the web.config that rewrites the robots.txt to robots_closed.txt for all urls ending with tamtam.nl
RobotsSharpParser
by:
nicholasb.za
.NET 6.0
18,954 total downloads
last updated
12/22/2021
Latest version:
2.0.0
robots
parse
robots
web
crawling
robots.txt
web
scraping
spider
sitemap
More tags
This client library enables working with Robots.txt. Key Features: - Parse robots.txt into Typed object. - Lookup Allowed/Disallowed/Crawldelay based on User-Agent. - Traverse...
More information
Louw.
SitemapParser
by:
louislouw
.NET 5.0
.NET Core 1.0
.NET Standard 1.6
.NET Framework 4.5.2
8,881 total downloads
last updated
1/6/2017
Latest version:
0.0.2
sitemaps.org
sitemap
index
sitemap
sitemap
parser
sitemap
reader
robots.txt
robots.txt
More tags
Louw.SitemapParser
ParkSquare.
AspNetCore.
Sitemap
by:
parksq
.NET 6.0
14,744 total downloads
last updated
12/30/2023
Latest version:
6.0.5
sitemap
robots
dotnetcore
asp.net
xml
search
generator
mvc
seo
googlebot
More tags
Automatic generation of sitemap.xml and robots.txt based on ASP.NET routes. Extensible engine allows custom URL providers to get URLs from any source. Includes exclude from sitemap feature and video sitemap...
More information
SeoTags
by:
mjebrahimi
.NET 5.0
.NET Core 2.1
6,081 total downloads
last updated
1/21/2022
Latest version:
2.0.1
SEO
OpenGraph
JSON-LD
SiteMap
Robots.txt
AspNetCore
MVC
RazorPages
HtmlHelper
SeoTags create all SEO tags you need such as meta, link, twitter card (twitter:), open graph (og:), and JSON-LD schema (structred data).
ChordsTech.
MultiSiteRobots
by:
ChordsTech
.NET Framework
1,143 total downloads
last updated
5/4/2020
Latest version:
1.0.0
C#
ASP.Net
Robots.txt
A simple lightweight .Net solution for websites which need easy management of multiple robots.txt
Toimik.
RobotsProtocol
by:
nurhafiz
.NET 8.0
3,995 total downloads
last updated
5/2/2024
Latest version:
0.5.0
robots.txt
robots-txt
robotstxt
robots-exclusion-standard
robots-exclusion-protocol
robots-meta-tag
x-robots-tag
robots
Parsers for Robots Exclusion Standard (aka robots.txt), Robots Meta Tag, and X-Robots-Tag. Visit Project Site for documentation.
Robots.
Txt.
Parser
by:
drmathias
.NET 7.0
594 total downloads
last updated
9/2/2023
Latest version:
1.0.0-rc8
Robots.Txt
Sitemap
Scraper
Crawler
Spider
Parses robots.txt files and sitemaps. Supports the proposed RFC9309 Robots Exclusion Protocol standard.
Nick.
RobotsParser
by:
nicholasb.za
.NET 6.0
4,033 total downloads
last updated
1/1/2023
Latest version:
2.0.6
robots
parse
robots
web
crawling
robots.txt
web
scraping
spider
sitemap
More tags
https://github.com/nicholasbergesen/RobotsParser/blob/master/README.md
WAGONN.
AspNetCore.
SEOHelper
by:
HasanAlbayrak
.NET 6.0
151 total downloads
last updated
2/13/2024
Latest version:
1.0.0
Robots.txt
sitemap.xml
SEO
Search Engine managemnet tools. forked AspNetCore.SEOHelper
CrawlCtrl
by:
CrawlCtrl
.NET 5.0
.NET Core 2.0
.NET Standard 2.0
.NET Framework 4.6.1
182 total downloads
last updated
1/21/2024
Latest version:
1.0.0-alpha.2
robots.txt
robots
crawler
crawling
sitemap
Deserialize robots.txt files
Sitemap.
NET
by:
LiveOrDevTrying
.NET 5.0
.NET Core 3.0
.NET Standard 2.1
9,751 total downloads
last updated
11/13/2021
Latest version:
2.1.5
sitemap
sitemap.xml
sitemap.net
sitemaps
sitemaps.net
robots
robot
robots.txt
SEO
search
More tags
A simple middleware built on reflection designed to support your Search Engine Optimized (SEO) app by dynamically creating a Sitemap.xml and Robots.txt. This package is designed to be very simple for simple use...
More information
Previous
Next