Digitals.Testing 0.3.0

Digitals.Testing

One package that brings the Digitals test stack, the shared test severity policy and a coverage gate to a .NET test project. Reference it instead of wiring up six test packages, copying an .editorconfig between repos and re-deciding what "enough coverage" means in every pipeline.

It is the companion to Digitals.Analyzers: that package covers production code, this one covers tests. Everything test-related lives here.

Install

<PackageReference Include="Digitals.Testing" />

In the test project, not in Directory.Build.props — the package configures the project it is referenced from, and that scoping is how the test policy stays off your production code.

No PrivateAssets here. This package ships real runtime assemblies (xunit, NSubstitute, the coverage gate), so a shared test-utilities library that packs itself needs them to flow on.

You also need the Microsoft.Testing.Platform runner selected in global.json at the repo root:

{
	"sdk": {
		"version": "10.0.100",
		"rollForward": "latestMinor"
	},
	"test": {
		"runner": "Microsoft.Testing.Platform"
	}
}

This is enforced: the build fails with DIGT001 when it is missing, because without it .NET 10's dotnet test routes through the VSTest target, no coverage is collected, and the coverage gate never runs — a green build that enforces nothing. Staging a migration? Opt out explicitly, with a comment saying why:

<DigitalsAllowVSTestRunner>true</DigitalsAllowVSTestRunner>

Then delete the packages this one replaces — Microsoft.NET.Test.Sdk, xunit.runner.visualstudio, coverlet.collector, xunit.v3, NSubstitute. Leaving them produces version conflicts, and the first four fail the build with DIGT002 anyway.

Migrating an existing repo: docs/ADOPTION.md.

What you get

Test data generators are deliberately not in that list. Bogus, AutoFixture and the rest are a per-repo choice, and a repo that wants one adds it in a line.

Plus configuration that is injected automatically — nothing to import:

• Project defaults: OutputType=Exe (xunit.v3 test projects are executables), IsTestProject=true, IsPackable=false, and the MTP entry point.
• global using Xunit;: shipped as a Using item, so no test project declares it. Works with or without ImplicitUsings, and the SDK deduplicates, so declaring it yourself as well is harmless. Opt out with <Using Remove="Xunit" />.
• Test severity policy at global_level 300, so it beats both of Digitals.Analyzers' configs and can relax rules that are right for production code and wrong for tests.
• Coverage collection, always on. dotnet test needs no flags and CI needs no bespoke invocation.
• A coverage gate that fails the run below the org floor, and a per-method CRAP limit.
• A TRX report in TestResults/, for the same reason: every results-publishing CI step reads one, and without it there is nothing to read. Turn it off per project with <DigitalsTrxReport>false</DigitalsTrxReport>.
• Custom rules (DT#####) — see below.
• Banned test packages (DIGT002, a build error) — see below.

Scoping, and why there is no .Tests convention here

Digitals.Analyzers applies to every project, so its test relaxations have to guess which projects are tests by matching the project name against .Tests. This package is referenced only by test projects, so the set of projects it applies to is the set of test projects. Name your test project whatever you like.

Coverage

Every dotnet test collects coverage into a Cobertura report and the gate judges it. The defaults are 100% line and 100% branch:

<DigitalsMinLineCoverage>100</DigitalsMinLineCoverage>
<DigitalsMinBranchCoverage>100</DigitalsMinBranchCoverage>

Below the floor, the run fails:

Line coverage 82.4% is below the 100.0% floor (416 of 505 covered). Add tests; do not lower the floor.

Both floors are a ratchet. An existing repo lands under them, so lower them once when you adopt the package, next to a comment naming what raises them back, and move them up as coverage climbs. Never down.

What is measured is set by Digitals.Testing.coverage.xml: test assemblies, auto-properties, [ExcludeFromCodeCoverage], generated code and EF migrations are all out. Point at your own file to override the policy:

<DigitalsCoverageSettingsFile>$(MSBuildThisFileDirectory)coverage.xml</DigitalsCoverageSettingsFile>

To turn the gate off while keeping the report:

<DigitalsSkipCoverageGate>true</DigitalsSkipCoverageGate>

Two things the gate does on purpose. It stays quiet when tests fail — a failing run has already said why, and coverage measured from a partial run is not a number worth judging. And it errors when it finds no report at all rather than passing: a gate that goes silent when it has no data is worse than no gate, because the build stays green while nothing is enforced.

Several reports are aggregated by summing covered and valid counts, then dividing once. Averaging the per-report rates would let a tiny uncovered assembly outweigh a large well-covered one.

The CRAP limit

The two floors are run-wide, so they cannot see the one method every bug lands in. CRAP (Change Risk Anti-Patterns) is judged per method instead:

complexity² × (1 − line coverage)³ + complexity

A fully covered method scores exactly its complexity; the same method with no tests scores complexity² + complexity. The cube is what makes the two interact, so covering a hairy method drops its score fast. The default limit is 30, which also caps complexity at 30 for anything fully covered.

Over the limit, the run fails and names the five worst:

1 method(s) score above the CRAP limit of 30.0: Lib.Calculator.Classify(int) scores 210.0 (complexity 14, 0 of 11 lines covered). Cover the branches or split the method; raising the limit is the last resort.

Same ratchet as the floors. Raise it once on adoption with a comment naming what brings it back down, then lower it:

<DigitalsMaxCrapScore>220</DigitalsMaxCrapScore>

Or take the check off and keep the floors:

<DigitalsSkipCrapGate>true</DigitalsSkipCrapGate>

Custom rules

Rule IDs here come from DT1####; Digitals.Analyzers keeps DT0####, so the two packages never collide in a #pragma or an .editorconfig.

DT10003 takes the trait's presence as policy and leaves the values to you, because a repo using Unit, Integration and Manual is as valid as one using two. Both halves are configurable:

# The trait name to require. Defaults to Category.
digitals_testing.required_trait = Category

# Optional. Unset means any non-empty value passes, which is the default.
digitals_testing.allowed_trait_values = Unit, Integration, Manual

DT10004 applies to everything in the project, fixtures and helpers included, since only test projects reference this package. An integration fixture whose retry loop genuinely needs wall-clock delay is the case to suppress, with a comment saying why.

Relaxed in tests

These are enforced by Digitals.Analyzers on production code and switched off here, because they fight the way tests are written: CA1822, CA1852, CA2201, CA1869, CA2012, CA1001, CA1859, S2344, S4144, S3881, S1172, S1186, S2701, IDE0060, VSTHRD200, VSTHRD002, VSTHRD003, CS8600, CS8604, CS8625.

The reason for each one is in Digitals.Testing.globalconfig.

Banned test packages

A direct PackageReference to any of these fails the build with DIGT002. Transitive dependencies are not checked.

The first four are new: they are the VSTest stack. Kept alongside this package they produce MTP0001 warnings, a coverage engine that never runs and a gate that silently enforces nothing.

If you genuinely need one, allow it explicitly with a comment saying why:

<DigitalsAllowBannedPackages>Moq</DigitalsAllowBannedPackages>

Turning something off

A whole rule, repo-wide. A repo-local .editorconfig always beats the packaged policy:

[*.cs]
dotnet_diagnostic.DT10001.severity = none

One line.

#pragma warning disable DT10001 // Named after the RFC section it covers
[Fact]
public void Rfc7231_Section_6_5_1() { }
#pragma warning restore DT10001

Suppressions are greppable and show up in review.

Notes

Build errors, not warnings, for the two structural checks: DIGT001 (MTP runner not selected) and DIGT002 (banned package). The DT##### rules ship as warning; how hard that bites is your repo's TreatWarningsAsErrors decision.

One version to review. Bumping xunit, NSubstitute or the coverage engine means a new version of this package, so an upgrade is a single version bump and a single changelog to read across every repo.

Versions come from git tags via MinVer and follow SemVer. Release notes: Releases.

License

MIT — see LICENSE. This covers the code in this repository: the config files, the banned-package check, the coverage gate and the custom DT##### analyzers. The bundled third-party packages keep their own licences and reach you as ordinary NuGet dependencies.

Working on the package itself: docs/MAINTAINING.md.