Darp.BinaryObjects 0.5.0

<div align="center">

Darp.BinaryObjects

A source generator to generate TryRead/Write Little/BigEndian methods for struct/class definitions.

</div>

[!IMPORTANT]
This package is under heavy development. Anything is subject to change.

You should use the source generation when you want:

• Serialization to a buffer of bytes
• Deserialization from a buffer already completely received
• Endianness during serialization
• Common interfaces for serialization are required which allow implementation of more complex scenarios by hand without the generator
• Usage of something like BinaryPrimitives but for more complex types
• Can work with a minimum c# LanguageVersion of 11 and net8.0 / net9.0

If these requirements do not meet your expectations, check out those other wonderful projects

• Several binary serializers. e.g. MemoryPack, BinaryPack, ... which are great if direct binary serialization is not needed
• Serialization libraries relying on reflection. e.g. HyperSerializer
• StructPacker - not supporting allocation less packing/unpacking
• BinarySerializer - Allows for binary serialization with a way larger feature set but more difficult to understand and relying on reflection

Supported properties

Here is a list of the property types currently supported by the library:

• Unmanaged types: bool, sbyte, byte, short, ushort, int, uint, long, ulong, char, float, double
• BinaryObjects implementing IBinaryWritable or IBinaryReadable
• Blittable types
• Enums
• Other .NET types: BitArray

For all of these types, it should be possible to define as array types:

• Memory abstractions: ReadOnlyMemory<T>
• Arrays: T[]
• Lists: List<T>
• Collections: IEnumerable<T>, IReadOnlyCollection<T>, ICollection<T>, IReadOnlyList<T>, IList<T>

To control these types there are attributes

• BinaryIgnore: Ignore some members
• BinaryElementCount: Sets the number of elements in an array
• BinaryReadRemaining: Reads the remaining into an array
• BinaryLength: Sets the length of a member
• BinaryConstantValue: Mark a (readonly) property which will have a predefined value

Unplanned:

• Unmanaged types have no clearly defined length / endianness: nint, nuint, decimal
• Multidimensional arrays (e.g. T[,], T[,,], etc.)
• Jagged arrays (e.g. T[][], etc.)
• Dictionaries: Dictionary<TKey, TValue>, IDictionary<TKey, TValue> and IReadOnlyDictionary<TKey, TValue>
• Nullable value types: Nullable<T> or T?

What is serialized?

• Any real, user-defined member in a class or struct declaration

• Any field or auto property which is settable or has a parameter with matching type and name in the constructor

• If there are multiple constructors defined the one with a BinaryConstructorAttribute is being used

There are warnings if:

• The constructor cannot be resolved
• There are multiple constructors but none with a BinaryConstructorAttribute
• A member is readonly and does not have a matching constructor argument or is explicitly ignored

How it's supposed to work

Let's pretend we have a series of bytes:

01020003040506

A: 01
B: 0200
Data: 03040506

Normally, you would have to write serialization methods for yourself. By adding the BinaryObjectAttribute, this is done automatically by the source generator.

This:

public readonly record struct SomeTestStruct(byte A, ushort B, ReadOnlyMemory<int> Data);

bool TryReadSomeTestStruct(ReadOnlySpan<byte> source, out SomeTestStruct value)
{
    if (source.Length < 3)
    {
        value = default;
        return false;
    }
    var a = source[0];
    var b = BinaryPrimitives.ReadUInt16LittleEndian(source[1..]);
    var dataArray = new int[(source.Length - 3) / sizeof(int)];
    ReadOnlySpan<int> reinterpretedData = MemoryMarshal.Cast<byte, int>(source[2..]);
    if (BitConverter.IsLittleEndian)
    {
        reinterpretedData.CopyTo(dataArray);
    }
    else
    {
        BinaryPrimitives.ReverseEndianness(reinterpretedData, dataArray);
    }
    value = new SomeTestStruct(a, b, dataArray);
    return true;
}

TryReadSomeTestStruct(buffer, out SomeTestStruct value);

Becomes this:

[BinaryObject]
public readonly partial record struct SomeTestStruct(byte A, ushort B, ReadOnlyMemory<int> Data);

SomeTestStruct.TryReadLittleEndian(buffer, out SomeTestStruct value);

Usage

// Define your object
[BinaryObject]
partial record struct YourStruct(ushort A, byte B);

// Read the struct from the buffer using either little or big endian format
var buffer = Convert.FromHexString("AABBCC");
var success = YourStruct.TryReadLittleEndian(source: buffer, out var value);
var success2 = YourStruct.TryReadBigEndian(source: buffer, out var value2, out int bytesRead);

// Get the actual size of the struct
var size = value.GetByteCount();

// Write the values back to a buffer
var writeBuffer = new byte[size];
var success3 = value.TryWriteLittleEndian(destination: writeBuffer);
var success4 = value2.TryWriteLittleEndian(destination: writeBuffer, out int bytesWritten);

The code generated by the struct will attempt to maximize readability by still maintaining performance and as little allocations as possible.

<details> <summary>Generated code</summary>

// <auto-generated/>
#nullable enable

using BinaryHelpers = global::Darp.BinaryObjects.BinaryHelpers;
using NotNullWhenAttribute = global::System.Diagnostics.CodeAnalysis.NotNullWhenAttribute;

namespace Your.Namespace;

/// <remarks> <list type="table">
/// <item> <term><b>Field</b></term> <description><b>Byte Length</b></description> </item>
/// <item> <term><see cref="A"/></term> <description>2</description> </item>
/// <item> <term><see cref="B"/></term> <description>1</description> </item>
/// <item> <term> --- </term> <description>3</description> </item>
/// </list> </remarks>
public partial record struct YourStruct : global::Darp.BinaryObjects.IBinaryWritable, global::Darp.BinaryObjects.IBinaryReadable<YourStruct>
{
    /// <inheritdoc />
    [global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.AggressiveInlining)]
    public int GetByteCount() => 3;

    /// <inheritdoc />
    public bool TryWriteLittleEndian(global::System.Span<byte> destination) => TryWriteLittleEndian(destination, out _);
    /// <inheritdoc />
    public bool TryWriteLittleEndian(global::System.Span<byte> destination, out int bytesWritten)
    {
        bytesWritten = 0;

        if (destination.Length < 3)
            return false;
        BinaryHelpers.WriteUInt16LittleEndian(destination[0..], this.A);
        BinaryHelpers.WriteUInt8(destination[2..], this.B);
        bytesWritten += 3;

        return true;
    }
    /// <inheritdoc />
    public bool TryWriteBigEndian(global::System.Span<byte> destination) => TryWriteBigEndian(destination, out _);
    /// <inheritdoc />
    public bool TryWriteBigEndian(global::System.Span<byte> destination, out int bytesWritten)
    {
        bytesWritten = 0;

        if (destination.Length < 3)
            return false;
        BinaryHelpers.WriteUInt16BigEndian(destination[0..], this.A);
        BinaryHelpers.WriteUInt8(destination[2..], this.B);
        bytesWritten += 3;

        return true;
    }

    /// <inheritdoc />
    public static bool TryReadLittleEndian(global::System.ReadOnlySpan<byte> source, out YourStruct value) => TryReadLittleEndian(source, out value, out _);
    /// <inheritdoc />
    public static bool TryReadLittleEndian(global::System.ReadOnlySpan<byte> source, out YourStruct value, out int bytesRead)
    {
        bytesRead = 0;
        value = default;

        if (source.Length < 3)
            return false;
        var ___readA = BinaryHelpers.ReadUInt16LittleEndian(source[0..]);
        var ___readB = BinaryHelpers.ReadUInt8(source[2..]);
        bytesRead += 3;

        value = new YourStruct(___readA, ___readB);
        return true;
    }
    /// <inheritdoc />
    public static bool TryReadBigEndian(global::System.ReadOnlySpan<byte> source, out YourStruct value) => TryReadBigEndian(source, out value, out _);
    /// <inheritdoc />
    public static bool TryReadBigEndian(global::System.ReadOnlySpan<byte> source, out YourStruct value, out int bytesRead)
    {
        bytesRead = 0;
        value = default;

        if (source.Length < 3)
            return false;
        var ___readA = BinaryHelpers.ReadUInt16BigEndian(source[0..]);
        var ___readB = BinaryHelpers.ReadUInt8(source[2..]);
        bytesRead += 3;

        value = new YourStruct(___readA, ___readB);
        return true;
    }
}

</details>

Development

After cloning the repository, you will find the following project structure:

• src/Darp.BinaryObjects contains public APIs and Attributes
• src/Darp.BinaryObjects.Generator contains the actual source generator
• test/Darp.BInaryObjects.Generator.Tests contains snapshot tests verifying the files generated by the source generator
• test/Darp.BinaryObjects.Tests contains unit tests ensuring the generated files actually valid

Code formatting

This repository uses CSharpier (inspired by prettier) for code formatting. CSharpier should be installed automatically when building the solution as a local dotnet tool.

To run it, execute

dotnet csharpier .

If you want to format you code on save, check out available Editor integration for your IDE.

Testing

Snapshot tests are done using Verify. If you want to optimize running these tests in your local IDE, you might adjust some settings. Please, check your local configuration in the VerifyDocs