MagicBytesValidator 2.2.0

MagicBytesValidator

Recognize file types from Streams or IFormFiles using MIME types or file extensions and validate them against the magic bytes according to the file types. The existing file types can be expanded in various ways.

How to install

• Install nuget package into your project:

Install-Package MagicBytesValidator -Version 2.2.0

dotnet add package MagicBytesValidator --version 2.2.0

• Reference in your csproj:

<PackageReference Include="MagicBytesValidator" Version="2.2.0" />

How to use

• Check if a stream content matches a file type:

  var validator = new MagicBytesValidator.Services.Validator();
  var pngFileType = validator.Mapping.FindByExtension("png");
  
  var isValidPng = await validator.IsValidAsync(memoryStream, pngFileType, CancellationToken.None);
  

• Check a file with its stream and file type:

var isValid = await validator.IsValidAsync(memoryStream, fileType, CancellationToken.None);

Validation strictness (FileByteType)

Some formats support multiple validation strategies (e.g. strict vs. lazy rules). For this purpose, the library exposes FileByteType:

• FileByteType.Strict (default)
• FileByteType.Lazy (optional relaxed/lazy rules for certain formats)

Validate an uploaded IFormFile with a specific validation type

The form file provider accepts an optional validationType parameter:

var fileType = await formFileTypeProvider.FindValidatedTypeAsync(
    formFile,
    null,
    CancellationToken.None,
    validationType: MagicBytesValidator.Models.FileByteType.Lazy
);

Example using Lazy (lazy rules if the format supports it):

var fileType = await formFileTypeProvider.FindValidatedTypeAsync(
    formFile,
    null,
    CancellationToken.None,
    validationType: MagicBytesValidator.Models.FileByteType.Lazy
);

Validate a stream with a specific validation type

The validator also accepts an optional validationType parameter:

var isValid = await validator.IsValidAsync(
    memoryStream,
    fileType,
    CancellationToken.None,
    validationType: MagicBytesValidator.Models.FileByteType.Strict
);

Example using Lazy:

var isValid = await validator.IsValidAsync(
    memoryStream,
    fileType,
    CancellationToken.None,
    validationType: MagicBytesValidator.Models.FileByteType.Lazy
);

Note: If a format does not define any Lazy-specific checks, Lazy behaves like “global checks only”. This keeps existing formats unchanged unless they opt into mode-specific rules.

Example: Lazy ("relaxed") PDF validation

Some PDFs contain additional trailing bytes after the %%EOF marker. While strict validation may require the file to end with %%EOF, lazy validation can accept the %%EOF marker anywhere within the last 1024 bytes of the file (behaviour tolerated by common PDF viewers).

Expand the file type mapping

• Get mapping:

// use the validator:
var mapping = validator.Mapping;

// use the formFileTypeProvider:
var mapping = formFileTypeProvider.Mapping;

// or create a new instance of the mapping:
var mapping = new MagicBytesValidator.Services.Mapping();

• Register a single FileByteFilter:

mapping.Register(
    new FileByteFilter(
        "traperto/trp", // MIME type
        new[] { "trp" } // file extensions
    ) {
        // magic byte sequences
        StartsWith(new byte?[]
        {
            0x78, 0x6c, 0x2f, 0x5f, 0x72, 0x65
        })
        .EndsWith(new byte?[]
        {
            0xFF, 0xFF
        })
    }
);

• FileByteFilters with specific offset checks:

mapping.Register(
    new FileByteFilter(
        "traperto/trp", // MIME type
        new[] { "trp" } // file extensions
    ) {
        // magic byte sequences
        Specific(new ByteCheck(512, new byte?[] { 0xFD }));
    }
);

ByteCheck allows for negative offset values to look for a specific offset counting from the end of file.

Optional: register mode-specific magic byte checks (Strict/Lazy)

When configuring a FileByteFilter, fluent methods accept an optional FileByteType parameter. If omitted, the check is global (applies to all validation types). If specified, the check applies only to that validation type.

Example:

StartsWith(new byte?[] { 0x25, 0x50, 0x44, 0x46, 0x2D }) // global
    .EndsWithAnyOf(new[]
    {
        new byte?[] { 0x25, 0x25, 0x45, 0x4F, 0x46 }
    }, MagicBytesValidator.Models.FileByteType.Strict)      // strict only
    .TailContains(1024, new byte?[] { 0x25, 0x25, 0x45, 0x4F, 0x46 },
        MagicBytesValidator.Models.FileByteType.Lazy);   // lazy only

var formFileTypeProvider = new MagicBytesValidator.Services.Http.FormFileTypeProvider();

try {
    var fileType = await formFileTypeProvider.FindValidatedTypeAsync(formFile, null, CancellationToken.None);

    if (fileType is not null) {
        // Further code
    } {
        // Can't determine type
    }
} catch(MagicBytesValidator.Exceptions.Http.MimeTypeMismatchException) {
    // Content and given MIME type / extension don't match.
}

// or create a new instance of the mapping: var mapping = new MagicBytesValidator.Services.Mapping();

  ```c#
  var streamFileTypeProvider = new MagicBytesValidator.Services.Streams.StreamFileTypeProvider();
  var fileType = await streamFileTypeProvider.TryFindUnambiguousAsync(fileStream, CancellationToken.None);

  if (fileType is not null) {
      // Further code
  } else {
      // Can't determine unambiguous type
  }

Add custom file types

• Register a custom type with filters:

  public class CustomType : MagicBytesValidator.Models.FileByteFilter
  {
      public CustomType() : base(
          ["traperto/trp"], // mime types
          ["trp"] // extensions
      )
      {
          // defined magic byte sequences
          StartsWith([
              0x78, 0x6c, 0x2f, 0x5f, 0x72, 0x65
          ])
          .EndsWith([
              0xFF, 0xFF
          ])
          .Specific(new ByteCheck(512, [0xFD])); // offset: 512 bytes, negative offset looks for a specific offset from the end of file
      }
  }
  
  var mapping = new MagicBytesValidator.Services.Mapping();
  mapping.Register(new CustomType());
  mapping.Register(new[] { new CustomType() }); // Add multiple types
  
  // Registering all `IFileType`s of the given assembly that are also not abstract and have an empty constructor.
  _mapping.Register(typeof(CustomType).Assembly);
  

CLI tool

There's a CLI tool (MagicBytesValidator.CLI) which can be used to determine MIME types for a local file by calling the following command:

dotnet run --project MagicBytesValidator.CLI -- [PATH]

This can be useful when debugging or validating newly added FileTypes.

List of file types

License

MIT License

▓▓  ▓▓▓▓▓▓▓▓▓
▓▓         ▓▓
▓▓▓▓▓▓▓▓▓  ▓▓
▓▓         ▓▓            traperto GmbH
▓▓  ▓▓▓▓▓▓▓▓▓
▓▓
▓▓▓▓▓▓▓▓▓   ▓▓