FractalDataWorks.Services.Transformations 0.4.0-preview.6

FractalDataWorks.Services.Transformations

Implementation package for the Transformations service domain in the FractalDataWorks framework. Provides data transformation infrastructure with ServiceType auto-discovery, factory-based creation, and provider-based service resolution.

Installation

<ProjectReference Include="..\FractalDataWorks.Services.Transformations\FractalDataWorks.Services.Transformations.csproj" />

Target Framework: net10.0

Service Registration (Three-Phase Pattern)

Transformations use a three-phase registration pattern for proper startup initialization:

var builder = WebApplication.CreateBuilder(args);

// Optional: Create logger for startup diagnostics (before DI is built)
using var loggerFactory = LoggerFactory.Create(b => b.AddConsole());

// Phase 1: Configure (binds IOptions from configuration)
TransformationTypes.Configure(builder.Services, builder.Configuration, loggerFactory);

// Phase 1: Register (adds factories to DI)
TransformationTypes.Register(builder.Services, loggerFactory);

var app = builder.Build();

// Phase 2: Initialize (optional, for eager loading)
TransformationTypes.Initialize(app.Services, loggerFactory);

app.Run();

Note: Initialize() is optional. When called, it eagerly resolves all factories at startup. When omitted, factories resolve lazily on first use.

Package Contents

This package contains:

• TransformationTypes - ServiceTypeCollection for transformation type discovery
• TransformationTypeBase<TService, TFactory, TConfiguration> - Base class for transformation types
• DefaultTransformationProvider - Provider implementation for transformation resolution
• TransformationEngine - Engine for executing transformation requests
• TransformationContext - Context for transformation operations
• TransformationRequest / TransformationResult - Request/response types
• TransformationsServiceBase<TCommand, TConfiguration, TService> - Base class for transformation services
• StandardTransformationServiceType - Built-in standard transformation type
• TransformationEngineLogger - MessageLogging integration

Core Types

TransformationTypes

The TransformationTypes class is a ServiceTypeCollection that provides auto-discovery and high-performance lookup of transformation types.

From TransformationTypes.cs:12-26:

[ExcludeFromCodeCoverage]
[ServiceTypeCollection(
    typeof(TransformationTypeBase<IGenericTransformation, ITransformationFactory<IGenericTransformation, ITransformationConfiguration>, ITransformationConfiguration>),
    typeof(ITransformationType),
    typeof(TransformationTypes),
    GenerateProvider = true,
    ServiceInterface = typeof(IGenericTransformation),
    ConfigurationInterface = typeof(ITransformationConfiguration),
    ProviderType = typeof(DefaultTransformationProvider),
    ProviderInterface = typeof(ITransformationProvider))]
public partial class TransformationTypes : ServiceTypeCollectionBase<
    TransformationTypeBase<IGenericTransformation, ITransformationFactory<IGenericTransformation, ITransformationConfiguration>, ITransformationConfiguration>,
    ITransformationType<IGenericTransformation, ITransformationConfiguration, ITransformationFactory<IGenericTransformation, ITransformationConfiguration>>>
{
}

DefaultTransformationProvider

The transformation provider uses DefaultServiceProviderBase to manage factories and create transformation instances from configuration. Configuration is loaded via IOptionsSnapshot<List<TransformationConfiguration>> which is bound to the database via MsSqlConfigurationSource.

TransformationTypeBase

Base class for defining transformation types with metadata about input/output types and capabilities.

From TransformationTypeBase.cs:27-33:

public abstract class TransformationTypeBase<TService, TFactory, TConfiguration> :
    ServiceTypeBase<TService, TFactory, TConfiguration, DefaultTransformationProvider>,
    ITransformationType, ITransformationType<TService, TConfiguration, TFactory>
    where TService : class, IGenericTransformation
    where TFactory : class, ITransformationFactory<TService, TConfiguration>
    where TConfiguration : class, ITransformationConfiguration
{

Key properties from TransformationTypeBase.cs:41-71:

/// <summary>
/// Gets the input type for this transformation.
/// </summary>
public Type InputType { get; }

/// <summary>
/// Gets the output type for this transformation.
/// </summary>
public Type OutputType { get; }

/// <summary>
/// Gets a value indicating whether this transformation supports streaming.
/// </summary>
public bool SupportsStreaming { get; }

/// <summary>
/// Gets the data container types supported by this transformation.
/// </summary>
public IDataContainerType[] SupportedContainers { get; }

Constructor from TransformationTypeBase.cs:233-247:

protected TransformationTypeBase(
    string name,
    Type inputType,
    Type outputType,
    bool supportsStreaming,
    IDataContainerType[] supportedContainers,
    OptionsLoaderBase? optionsLoader = null,
    string? category = null)
    : base(name, $"Transformations:{name}", $"{name} Transformation", $"Transformation service for {inputType.Name} to {outputType.Name}", optionsLoader ?? OptionsLoaderTypes.Reloadable, category ?? "Transformation")
{
    InputType = inputType ?? throw new ArgumentNullException(nameof(inputType));
    OutputType = outputType ?? throw new ArgumentNullException(nameof(outputType));
    SupportsStreaming = supportsStreaming;
    SupportedContainers = supportedContainers ?? throw new ArgumentNullException(nameof(supportedContainers));
}

StandardTransformationServiceType

Built-in transformation type for standard transformation operations.

From StandardTransformationServiceType.cs:15-31:

[ServiceTypeOption(typeof(TransformationTypes), "StandardTransformation")]
public sealed class StandardTransformationServiceType :
    TransformationTypeBase<IGenericTransformation, ITransformationFactory<IGenericTransformation, ITransformationConfiguration>, ITransformationConfiguration>
{
    public StandardTransformationServiceType()
        : base(
            name: "StandardTransformation",
            inputType: typeof(object),
            outputType: typeof(object),
            supportsStreaming: true,
            supportedContainers: [] // FUTURE: Add actual container types
            )
    {
    }

TransformationEngine

The engine manages transformation execution with start/stop lifecycle and provider resolution.

From TransformationEngine.cs:17-35:

public sealed class TransformationEngine : ITransformationEngine
{
    private readonly ILogger<TransformationEngine> _logger;
    private readonly IServiceProvider _serviceProvider;
    private bool _isRunning;

    public TransformationEngine(
        ILogger<TransformationEngine> logger,
        IServiceProvider serviceProvider)
    {
        _logger = logger ?? throw new ArgumentNullException(nameof(logger));
        _serviceProvider = serviceProvider ?? throw new ArgumentNullException(nameof(serviceProvider));
        EngineId = Guid.NewGuid().ToString();
        EngineType = "TransformationEngine";
        _isRunning = false;
    }

    public string EngineId { get; }
    public string EngineType { get; }
    public bool IsRunning => _isRunning;

Start/stop lifecycle from TransformationEngine.cs:125-150:

public Task<IGenericResult> Start(CancellationToken cancellationToken = default)
{
    if (_isRunning)
    {
        var message = TransformationEngineLogger.EngineAlreadyRunning(_logger);
        return Task.FromResult(GenericResult.Success(message));
    }

    _isRunning = true;
    TransformationEngineLogger.EngineStarted(_logger, EngineId);
    return Task.FromResult(GenericResult.Success());
}

public Task<IGenericResult> Stop(CancellationToken cancellationToken = default)
{
    if (!_isRunning)
    {
        var message = TransformationEngineLogger.EngineNotRunning(_logger);
        return Task.FromResult(GenericResult.Success(message));
    }

    _isRunning = false;
    TransformationEngineLogger.EngineStopped(_logger, EngineId);
    return Task.FromResult(GenericResult.Success());
}

TransformationContext

Context for transformation operations with correlation and metadata support.

From TransformationContext.cs:14-50:

[ExcludeFromCodeCoverage]
public sealed class TransformationContext : ITransformationContext
{
    private readonly Dictionary<string, object> _properties;

    public TransformationContext(
        string? identity = null,
        string? correlationId = null,
        string? pipelineStage = null,
        IReadOnlyDictionary<string, object>? properties = null)
    {
        Identity = identity;
        CorrelationId = correlationId ?? Guid.NewGuid().ToString();
        PipelineStage = pipelineStage;
        _properties = properties != null
            ? new Dictionary<string, object>(properties, StringComparer.Ordinal)
            : new Dictionary<string, object>(StringComparer.Ordinal);
    }

    public string? Identity { get; }
    public string? CorrelationId { get; }
    public string? PipelineStage { get; }
    public IReadOnlyDictionary<string, object> Properties => _properties;
}

TransformationRequest

Request type for transformation operations with immutable transformation support.

From TransformationRequest.cs:13-66:

[ExcludeFromCodeCoverage]
public class TransformationRequest : ITransformationRequest
{
    public TransformationRequest()
    {
        CommandId = Guid.NewGuid();
        RequestId = CommandId;
        CommandType = "TransformationRequest";
        CreatedAt = DateTime.UtcNow;
        Configuration = new Dictionary<string, object>(StringComparer.Ordinal);
        Options = new Dictionary<string, object>(StringComparer.Ordinal);
    }

    public Guid CommandId { get; init; }
    public string CommandType { get; init; }
    public string Category { get; init; } = "Transformation";
    public DateTime CreatedAt { get; init; }
    public Guid RequestId { get; init; }
    public object? InputData { get; init; }
    public string InputType { get; init; } = string.Empty;
    public string OutputType { get; init; } = string.Empty;
    public string? TransformationCategory { get; init; }
    public IReadOnlyDictionary<string, object> Configuration { get; init; }
    public IReadOnlyDictionary<string, object> Options { get; init; }
    public TimeSpan? Timeout { get; init; }
    public ITransformationContext? Context { get; init; }
    public Type ExpectedResultType { get; init; } = typeof(object);

Immutable transformation methods from TransformationRequest.cs:71-87:

public ITransformationRequest WithInputData(object? newInputData, string? newInputType = null)
{
    return new TransformationRequest
    {
        RequestId = RequestId,
        Category = Category,
        InputData = newInputData,
        InputType = newInputType ?? InputType,
        OutputType = OutputType,
        TransformationCategory = TransformationCategory,
        Configuration = Configuration,
        Options = Options,
        Timeout = Timeout,
        Context = Context,
        ExpectedResultType = ExpectedResultType
    };
}

TransformationsServiceBase

Base class for building transformation service implementations.

From Services/TransformationsServiceBase.cs:20-35:

public abstract class TransformationsServiceBase<TCommand, TConfiguration, TService>
    : ServiceBase<TCommand, TConfiguration, TService>
    where TCommand : ITransformationsCommand
    where TConfiguration : class, ITransformationConfiguration
    where TService : class
{
    protected TransformationsServiceBase(ILogger<TService> logger, TConfiguration configuration)
        : base(logger, configuration)
    {
    }
}

MessageLogging Integration

The package uses MessageLogging for structured logging that returns messages for Railway-Oriented Programming.

From Logging/TransformationEngineLogger.cs:12-51:

public static partial class TransformationEngineLogger
{
    [MessageLogging(
        EventId = 6000,
        Level = LogLevel.Information,
        Message = "Transformation engine started: {engineId}")]
    public static partial IGenericMessage EngineStarted(ILogger logger, string engineId);

    [MessageLogging(
        EventId = 6001,
        Level = LogLevel.Warning,
        Message = "Transformation engine is already running")]
    public static partial IGenericMessage EngineAlreadyRunning(ILogger logger);

    [MessageLogging(
        EventId = 6002,
        Level = LogLevel.Information,
        Message = "Transformation engine stopped: {engineId}")]
    public static partial IGenericMessage EngineStopped(ILogger logger, string engineId);

    [MessageLogging(
        EventId = 6004,
        Level = LogLevel.Information,
        Message = "Executing transformation: RequestId={requestId}, InputType={inputType}, OutputType={outputType}, Category={category}")]
    public static partial IGenericMessage ExecutingTransformation(
        ILogger logger,
        string requestId,
        string inputType,
        string outputType,
        string category);

Configuration

Transformations are configured in the Transformations section of appsettings. The provider reads the TransformationType discriminator to determine which factory to use.

From DefaultTransformationProvider.cs:66-69:

public string? GetTypeName(string configurationName)
{
    return _configuration[$"Transformations:{configurationName}:TransformationType"];
}

Example configuration structure:

{
  "Transformations": {
    "MyTransformation": {
      "TransformationType": "StandardTransformation",
      "Name": "MyTransformation"
    }
  }
}

Available Transformation Packages

Dependencies

This package depends on:

• FractalDataWorks.Services.Transformations.Abstractions - Transformation contracts
• FractalDataWorks.Services.Abstractions - Service infrastructure
• FractalDataWorks.Services.Abstractions - ServiceType infrastructure
• FractalDataWorks.Results - Railway-Oriented Programming
• FractalDataWorks.Services - Base service implementations
• FractalDataWorks.Abstractions - Core abstractions
• FractalDataWorks.Messages - Messaging infrastructure
• FractalDataWorks.MessageLogging.Abstractions - MessageLogging attributes
• Microsoft.Extensions.DependencyInjection.Abstractions - DI abstractions
• Microsoft.Extensions.Logging.Abstractions - Logging abstractions

Related Packages

• FractalDataWorks.Services.Transformations.Abstractions - Interface contracts
• FractalDataWorks.Services.Transformations.DataCleaning - Data cleaning transformations
• FractalDataWorks.Services.Abstractions - Service infrastructure contracts
• FractalDataWorks.Services.Abstractions - ServiceType infrastructure