FractalDataWorks.Schema.Abstractions 0.4.0-preview.6

FractalDataWorks.Schema.Abstractions

Unified schema abstractions for property classification and data layouts.

Overview

This project provides the foundational types for describing data structures in a storage-agnostic way. It introduces a PropertyRole-based type system that eliminates switch statements and enables type-safe schema definitions.

Key Concepts

PropertyRoles - Type-Safe Property Classification

PropertyRoles replace magic strings and enums with an extensible MutableTypeCollection:

Available Roles:

• Surrogate - Auto-generated key with no business meaning (e.g., Id, RowId)
• NaturalKey - Business identifier, human-meaningful (e.g., Email, SSN, composite keys)
• Lookup - Indexed for search, not part of key (e.g., LastName, ZipCode)
• Attribute - Descriptive, non-indexed (e.g., Description, Notes)
• Measure - Aggregatable numeric (e.g., Price, Quantity, TotalSales)

Usage Example:

using FractalDataWorks.Schema;
using FractalDataWorks.Schema.Properties;

var property = new PropertyDefinition
{
    Name = "CustomerId",
    Role = PropertyRoles.NaturalKey,  // Type-safe! No magic strings!
    DataType = DataTypes.Integer,
    IsRequired = true
};

// No switch statements - just property access!
if (property.Role.IsKeyRole)
{
    GeneratePrimaryKeyConstraint(property);
}
if (property.Role.IsIndexable)
{
    GenerateIndex(property);
}

DataLayouts - Storage Structure Classification

DataLayouts describe how data is physically organized:

Available Layouts:

• Tabular - Row/column structure (SQL tables, CSV)
• Hierarchical - Parent/child relationships (JSON, XML)
• Document - Nested key-value pairs (MongoDB, JSON documents)
• KeyValue - Simple key-value pairs (Redis, DynamoDB)
• Graph - Node/edge relationships (Neo4j, CosmosDB Graph)

Usage Example:

using FractalDataWorks.Schema;

var schema = new SchemaDefinition<ColumnDefinition>
{
    Name = "Customers",
    Layout = DataLayouts.Tabular,  // SQL table
    Properties = [...]
};

if (schema.Layout.SupportsNesting)
{
    // Add child schemas for nested structures
}

ISchemaDefinition - Unified Schema Interface

Provides a storage-agnostic way to describe data structures:

public interface ISchemaDefinition<TProperty> where TProperty : IPropertyDefinition
{
    string Name { get; }
    IReadOnlyList<TProperty> Properties { get; }
    IKeyDefinition<TProperty>? SurrogateKey { get; }
    IKeyDefinition<TProperty>? NaturalKey { get; }
    IReadOnlyList<IIndexDefinition<TProperty>> Indexes { get; }
    IDataLayout Layout { get; }
    IReadOnlyList<ISchemaDefinition<TProperty>>? Children { get; }
}

Key Features:

• Supports both surrogate keys (auto-generated) and natural keys (business identifiers)
• Allows composite keys via IKeyDefinition<TProperty>
• Supports hierarchical schemas via Children property
• Layout-aware for format-specific operations

Property Definition Types

IPropertyDefinition (Base)

Generic interface for all property types.

ColumnDefinition (Physical)

Describes a physical database column:

var column = new ColumnDefinition
{
    Name = "CustomerId",
    Role = PropertyRoles.NaturalKey,
    DataType = DataTypes.Integer,
    MaxLength = null,
    IsRequired = true,
    DefaultValue = null
};

FieldDefinition (Logical)

Describes a logical field in a document or message:

var field = new FieldDefinition
{
    Name = "email",
    Role = PropertyRoles.Lookup,
    DataType = DataTypes.String,
    PathExpression = "$.customer.email"
};

Key Definitions

using FractalDataWorks.Schema.Keys;

// Surrogate key (single-column)
var surrogateKey = new KeyDefinition<ColumnDefinition>
{
    Name = "PK_Customer",
    Properties = [idColumn],
    IsPrimary = true
};

// Natural key (composite)
var naturalKey = new KeyDefinition<ColumnDefinition>
{
    Name = "UK_Customer_Email",
    Properties = [emailColumn, tenantIdColumn],
    IsPrimary = false
};

Index Definitions

using FractalDataWorks.Schema.Indexes;

var index = new IndexDefinition<ColumnDefinition>
{
    Name = "IX_Customer_LastName",
    Properties = [lastNameColumn, firstNameColumn],
    IsUnique = false,
    IsClustered = false,
    FilterExpression = "WHERE IsActive = 1"
};

Integration with DataSets and DataContainers

DataSet (Logical) → Uses FieldDefinition, focuses on WHAT the data represents DataContainer (Physical) → Uses ColumnDefinition, focuses on HOW data is stored

// DataSet schema (logical)
var dataSetSchema = new SchemaDefinition<FieldDefinition>
{
    Name = "CustomerData",
    Layout = DataLayouts.Hierarchical,
    Properties = [
        new FieldDefinition { Name = "id", Role = PropertyRoles.NaturalKey },
        new FieldDefinition { Name = "name", Role = PropertyRoles.Attribute },
        new FieldDefinition { Name = "email", Role = PropertyRoles.Lookup }
    ]
};

// DataContainer schema (physical)
var containerSchema = new SchemaDefinition<ColumnDefinition>
{
    Name = "Customer",
    Layout = DataLayouts.Tabular,
    Properties = [
        new ColumnDefinition { Name = "Id", Role = PropertyRoles.Surrogate },
        new ColumnDefinition { Name = "Name", Role = PropertyRoles.Attribute },
        new ColumnDefinition { Name = "Email", Role = PropertyRoles.NaturalKey }
    ],
    SurrogateKey = new KeyDefinition<ColumnDefinition> { ... },
    Indexes = [...]
};

Why PropertyRoles Instead of Enums?

Traditional Approach (BAD):

enum PropertyRole { Surrogate, NaturalKey, Lookup, Attribute, Measure }

// Leads to switch statements everywhere!
switch (property.Role)
{
    case PropertyRole.Surrogate:
    case PropertyRole.NaturalKey:
        GeneratePrimaryKey(property);
        break;
}

FDW Approach (GOOD):

// No switch - behavior is on the role itself!
if (property.Role.IsKeyRole)
{
    GeneratePrimaryKey(property);
}

Benefits:

• Extensible - Add new roles in downstream projects via [TypeOption]
• No Switch Statements - Roles carry their own behavior
• Type-Safe - Compile-time discovery via source generators
• Cross-Assembly - Roles can be defined anywhere

Dependencies

• FractalDataWorks.Collections - TypeCollection base classes
• FractalDataWorks.Collections.SourceGenerators - Source generator for PropertyRoles and DataLayouts

Target Framework

netstandard2.0 - Maximum compatibility with downstream projects.

See Also

• FractalDataWorks.Schema.Ddl - DDL command interfaces for schema-to-SQL generation
• FractalDataWorks.Schema.Ddl.MsSql - SQL Server DDL generator implementation
• FractalDataWorks.Data.DataSets.Abstractions - Logical data definitions (uses FieldDefinition)
• FractalDataWorks.Data.DataContainers.Abstractions - Physical schema definitions (uses ColumnDefinition)