FractalDataWorks.Data.Abstractions 0.4.0-preview.6

FractalDataWorks.Data.Abstractions

Core data abstraction layer providing contracts for data operations across formats, schemas, containers, and transformations.

Overview

FractalDataWorks.Data.Abstractions defines the core contracts for data access patterns:

• Data Format Abstractions - Format types for data serialization (Tabular, JSON, XML, CSV)
• Schema Contracts - Field definitions with Identity/Attribute/Measure roles
• Container Abstractions - Storage containers with format and schema metadata
• Path Abstractions - Navigation paths within data stores
• Converter Abstractions - Type conversion between data formats with DbType support
• Transformer Abstractions - ETL-style data transformation contracts

Target Frameworks: .NET Standard 2.0, .NET 10.0

Dependencies: FractalDataWorks.Collections, FractalDataWorks.Results

Key Components

Data Formats

From IFormatType.cs:11-42:

public interface IFormatType : ITypeOption<int>
{
    string ConfigurationKey { get; }
    string DisplayName { get; }
    string Description { get; }
    string MimeType { get; }
    bool IsBinary { get; }
    bool SupportsStreaming { get; }
}

From FormatTypes.cs:12-16:

[TypeCollection(typeof(FormatTypeBase), typeof(IFormatType), typeof(FormatTypes), RestrictToCurrentCompilation = false)]
public sealed partial class FormatTypes : TypeCollectionBase<FormatTypeBase, IFormatType>
{
    // TypeCollectionGenerator will generate all members
}

Available format types (from TypeOption implementations):

• FormatTypes.Tabular - SQL result sets, in-memory tables
• FormatTypes.Json - JavaScript Object Notation
• FormatTypes.Xml - XML data format
• FormatTypes.Csv - Comma-Separated Values

Field Types

From IFieldType.cs:9-25:

public interface IFieldType
{
    string TypeName { get; }
    Type ClrType { get; }
    bool IsNested { get; }
}

From ISimpleFieldType.cs:6-9:

public interface ISimpleFieldType : IFieldType
{
    // Marker interface - no additional members
}

From IArrayFieldType.cs:6-12:

public interface IArrayFieldType : IFieldType
{
    IFieldType ElementType { get; }
}

From IObjectFieldType.cs:8-14:

public interface IObjectFieldType : IFieldType
{
    IReadOnlyList<IField> Fields { get; }
}

Schema System

From FieldRole.cs:6-25:

public enum FieldRole
{
    Identity,   // Part of unique key, used for joins and lookups (UserId, OrderId)
    Attribute,  // Descriptive/dimensional data for filtering and grouping (CustomerName, OrderDate)
    Measure     // Numeric/aggregatable facts for SUM, AVG, COUNT operations (TotalAmount, Quantity)
}

From IField.cs:6-57:

public interface IField
{
    string Name { get; }
    IFieldType FieldType { get; }
    FieldRole Role { get; }
    bool IsNullable { get; }
    string? Description { get; }
    string? TypeSystemId { get; }
    int? ConverterTypeId { get; }
    bool IsPrimaryKey { get; }
    bool IsIdentity { get; }
    bool IsComputed { get; }
}

From IContainerSchema.cs:9-40:

public interface IContainerSchema
{
    IReadOnlyList<IField> Fields { get; }
    IReadOnlyList<IField> GetIdentityFields();
    IReadOnlyList<IField> GetAttributeFields();
    IReadOnlyList<IField> GetMeasureFields();
    IField? GetField(string name);
    bool SupportsNesting { get; }
}

Container System

From IStorageContainer.cs:17-68:

public interface IStorageContainer
{
    string Name { get; }
    IContainerType ContainerType { get; }
    IFormatType Format { get; }
    IContainerSchema Schema { get; }
    IPath Path { get; }
    string[] SupportedOperations { get; }
    IReadOnlyDictionary<string, object> Metadata { get; }
}

From ContainerTypes.cs:12-16:

[TypeCollection(typeof(ContainerTypeBase), typeof(IContainerType), typeof(ContainerTypes), RestrictToCurrentCompilation = false)]
public sealed partial class ContainerTypes : TypeCollectionBase<ContainerTypeBase, IContainerType>
{
    // TypeCollectionGenerator will generate all members
}

Available container types (from TypeOption implementations in downstream packages):

• ContainerTypes.Table - SQL database tables (FractalDataWorks.Data.MsSql)
• ContainerTypes.View - SQL database views (FractalDataWorks.Data.MsSql)
• ContainerTypes.StoredProcedure - SQL stored procedures (FractalDataWorks.Data.MsSql)
• ContainerTypes.Endpoint - HTTP/REST endpoints (FractalDataWorks.Data.Http)
• ContainerTypes.File - File system files (FractalDataWorks.Data.Files)

Path System

From IPath.cs:6-17:

public interface IPath
{
    string PathValue { get; }
    string Domain { get; }  // Sql, Rest, File, GraphQL
}

From IDataPath.cs:22-139:

public interface IDataPath
{
    string Id { get; }
    string Name { get; }
    string PathType { get; }
    string FullPath { get; }
    IReadOnlyList<string> Segments { get; }
    IReadOnlyDictionary<string, PathParameter> Parameters { get; }
    IReadOnlyDictionary<string, object> Metadata { get; }
    bool RequiresParameters { get; }
    IDataPath ResolveParameters(IDictionary<string, object> parameters);
    IGenericResult ValidateParameters(IDictionary<string, object> parameters);
    IDataPath? GetParent();
    IEnumerable<IDataPath> GetChildren();
    IDataPath Combine(string relativePath);
}

Type Converters

From IDataTypeConverter.cs:15-63:

public interface IDataTypeConverter : ITypeOption<int, DataTypeConverterBase>
{
    string SourceType { get; }
    Type TargetClrType { get; }
    DbType DbType { get; }
    object? ToClr(object? dbValue);
    object? ToDb(object? clrValue);
    int? Size { get; }
    byte? Precision { get; }
    byte? Scale { get; }
}

From DataTypeConverters.cs:13-26:

[TypeCollection(typeof(DataTypeConverterCollectionBase),
                typeof(IDataTypeConverters),
                typeof(DataTypeConverters))]
public abstract partial class DataTypeConverters
    : TypeCollectionBase<DataTypeConverterCollectionBase, IDataTypeConverters>
{
    // Source generator creates child collection accessors like:
    // - public static IDataTypeConverters MsSql { get; }
}

Data Transformers

From IDataTransformer.cs:10-36:

public interface IDataTransformer
{
    string TransformerName { get; }
}

public interface IDataTransformer<TResult, TInput> : IDataTransformer
{
    Task<IGenericResult<TResult>> Transform(
        TInput input,
        CancellationToken cancellationToken = default);
}

From DataTransformerTypes.cs:12-16:

[TypeCollection(typeof(DataTransformerTypeBase), typeof(IDataTransformerType), typeof(DataTransformerTypes), RestrictToCurrentCompilation = false)]
public sealed partial class DataTransformerTypes : TypeCollectionBase<DataTransformerTypeBase, IDataTransformerType>
{
    // TypeCollectionGenerator will generate all members
}

Usage Examples

Creating a Field Definition

From Field.cs:6-57:

var idField = new Field
{
    Name = "UserId",
    FieldType = new SimpleFieldType { TypeName = "Int32", ClrType = typeof(int) },
    Role = FieldRole.Identity,
    IsNullable = false,
    IsPrimaryKey = true
};

var emailField = new Field
{
    Name = "Email",
    FieldType = new SimpleFieldType { TypeName = "String", ClrType = typeof(string) },
    Role = FieldRole.Attribute,
    IsNullable = false
};

var amountField = new Field
{
    Name = "TotalAmount",
    FieldType = new SimpleFieldType { TypeName = "Decimal", ClrType = typeof(decimal) },
    Role = FieldRole.Measure,
    IsNullable = false
};

Creating a Container Schema

From ContainerSchema.cs:10-46:

var schema = new ContainerSchema
{
    Fields = new List<IField> { idField, emailField, amountField }
};

// Query fields by role
var identityFields = schema.GetIdentityFields();   // [UserId]
var attributeFields = schema.GetAttributeFields(); // [Email]
var measureFields = schema.GetMeasureFields();     // [TotalAmount]

// Check for nested types
bool hasNesting = schema.SupportsNesting;

Using Format Types

// Access format types via TypeCollection lookups
var jsonFormat = FormatTypes.ByName("Json");
var csvFormat = FormatTypes.ByName("Csv");
var tabularFormat = FormatTypes.ByName("Tabular");

// Check format properties
bool isBinary = jsonFormat.IsBinary;           // false
bool supportsStreaming = jsonFormat.SupportsStreaming; // true
string mimeType = jsonFormat.MimeType;         // "application/json"

Best Practices

1. Use FieldRole semantically - Identity for keys, Attribute for descriptive data, Measure for aggregatable values
2. Prefer IContainerSchema methods - Use GetIdentityFields(), GetAttributeFields(), GetMeasureFields() for role-based queries
3. Use TypeCollection lookups - Access types via FormatTypes.ByName() or static properties
4. Check SupportsNesting - Verify schema capabilities before processing hierarchical data
5. Use IGenericResult - All transformer operations return Result types for error handling

Dependencies

• FractalDataWorks.Collections - TypeCollections infrastructure
• FractalDataWorks.Results - Result pattern support

Related Packages

• FractalDataWorks.Data - Core data layer implementations
• FractalDataWorks.Data.MsSql - SQL Server container types (Table, View, StoredProcedure)
• FractalDataWorks.Data.Http - HTTP container types (Endpoint)
• FractalDataWorks.Data.Files - File system container types (File)