Flowthru 0.1.5

There is a newer version of this package available.
See the version list below for details.

dotnet add package Flowthru --version 0.1.5

NuGet\Install-Package Flowthru -Version 0.1.5

This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.

<PackageReference Include="Flowthru" Version="0.1.5" />

For projects that support PackageReference, copy this XML node into the project file to reference the package.

<PackageVersion Include="Flowthru" Version="0.1.5" />
                    

                            Directory.Packages.props

<PackageReference Include="Flowthru" />
                    

                            Project file

For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.

paket add Flowthru --version 0.1.5

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

#r "nuget: Flowthru, 0.1.5"

#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.

#:package Flowthru@0.1.5

#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.

#addin nuget:?package=Flowthru&version=0.1.5
                    

                            Install as a Cake Addin

#tool nuget:?package=Flowthru&version=0.1.5
                    

                            Install as a Cake Tool

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

Flowthru

A type-safe data engineering framework for .NET inspired by Kedro's declarative pipelines.

Version: 0.1.5 (Alpha - Simplified AddNode API)
Status: Active Development

Quick Start

# Install
dotnet add package Flowthru

# Create your catalog
public class MyCatalog : DataCatalogBase
{
    public CsvCatalogDataset<RawData> RawData { get; }
    public ParquetCatalogDataset<ProcessedData> ProcessedData { get; }
    
    public MyCatalog()
    {
        RawData = CreateCsvDataset<RawData>("raw", "data/raw.csv");
        ProcessedData = CreateParquetDataset<ProcessedData>("processed", "data/processed.parquet");
    }
}

// Run your application (inline registration)
public class Program
{
    public static Task<int> Main(string[] args)
    {
        return FlowthruApplication.Create(args, builder =>
        {
            builder.UseCatalog(new MyCatalog());
            
            // Register pipelines directly in the builder
            builder
                .RegisterPipeline<MyCatalog>("data_processing", DataProcessingPipeline.Create)
                .WithDescription("Process raw data")
                .WithTags("etl");
        });
    }
}

# Execute
dotnet run data_processing

Documentation

Tutorials

TODO: Improve View Spaceflights test pipeline

Explanation

Understanding-oriented: Why Flowthru works the way it does.

Why Compile-Time Safety Matters
Understanding the Three Compile-Time Safety Layers
Common Pitfalls and How to Avoid Them
Design Rationale: Why These Choices?
Comparing with Kedro: Key Differences
When to Use Flowthru vs Kedro
Future Enhancements

Core Features

✅ Compile-Time Type Safety - Catch errors before execution, not during
✅ Configuration File Support - JSON/YAML configs with environment layering
✅ Eager Schema Validation - Validate external data before pipeline runs (optional)
✅ Read/Write Capability Enforcement - Prevent writes to read-only sources (Excel, APIs)
✅ Generic Pipeline Registry - Type-safe pipeline registration with parameters
✅ Property-Based Catalogs - IntelliSense-driven dataset access
✅ Expression-Based Mapping - Refactoring-safe multi-input/output nodes
✅ Application Builder Pattern - Minimal boilerplate (~15 lines)
✅ Formatted Console Output - Beautiful execution logs with progress tracking
✅ Dependency Injection - Property injection for services and loggers

Philosophy

Fail at compile-time, not runtime.

Every design decision in Flowthru prioritizes catching errors during compilation:

Typos in catalog keys → Compile error (property doesn't exist)
Wrong data types → Compile error (generic constraint violation)
Missing parameters → Compile error (property not found)
Type mismatches → Compile error (incompatible types)

The C# compiler becomes your documentation, showing all available datasets, parameters, and types through IntelliSense.

Comparison with Kedro

Aspect	Kedro	Flowthru
Type Safety	Runtime (duck typing)	Compile-time (generics)
Catalog	YAML + string keys	C# properties
Nodes	Functions with decorators	Classes inheriting `NodeBase<TIn,TOut>`
Parameters	YAML dictionaries	Strongly-typed record classes
Error Detection	During execution	During compilation
IDE Support	Basic	Full (IntelliSense, refactoring, navigation)

Requirements

.NET 9.0 or later
C# 12 or later

License

MIT License - See LICENSE file for details

Acknowledgments

Kedro: Inspiration for declarative pipeline patterns
Microsoft.Extensions: DI and logging infrastructure

Product	Compatible and additional computed target framework versions.
.NET	net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed.

Compatible target framework(s)

Included target framework(s) (in package)

Learn more about Target Frameworks and .NET Standard.

net10.0
- CsvHelper (>= 30.0.1)
- ExcelDataReader (>= 3.6.0)
- ExcelDataReader.DataSet (>= 3.6.0)
- LanguageExt.Core (>= 5.0.0-beta-54)
- Microsoft.Extensions.Configuration (>= 9.0.10)
- Microsoft.Extensions.Configuration.Binder (>= 9.0.10)
- Microsoft.Extensions.Configuration.Json (>= 9.0.10)
- Microsoft.Extensions.DependencyInjection (>= 9.0.10)
- Microsoft.Extensions.DependencyInjection.Abstractions (>= 9.0.10)
- Microsoft.Extensions.Logging (>= 9.0.10)
- Microsoft.Extensions.Logging.Abstractions (>= 9.0.10)
- Microsoft.Extensions.Logging.Console (>= 9.0.10)
- NetEscapades.Configuration.Yaml (>= 3.1.0)
- Parquet.Net (>= 5.2.0)
- System.ComponentModel.Annotations (>= 5.0.0)

NuGet packages (6)

Showing the top 5 NuGet packages that depend on Flowthru:

Package	Downloads
Flowthru.Extensions.ML Pure, unoptimized UMAP implementation for Flowthru - direct port from Python reference implementation by Leland McInnes	1.1K
Flowthru.Integrations.MLNet ML.NET integration for Flowthru - provides IDataView container adapters for machine learning pipelines	1.1K
Flowthru.Misc.ML Pure, unoptimized UMAP implementation for Flowthru - direct port from Python reference implementation by Leland McInnes	195
Flowthru.Extensions.MLNet ML.NET integration for Flowthru - provides IDataView container adapters for machine learning pipelines	103
Flowthru.Extensions.EFCore Entity Framework Core integration for Flowthru data catalogs	99

GitHub repositories

This package is not used by any popular GitHub repositories.

Version	Downloads	Last Updated
0.1.24	47	3/9/2026
0.1.23	48	3/9/2026
0.1.22	63	3/6/2026
0.1.16	90	3/5/2026
0.1.15	151	3/2/2026
0.1.14	109	2/28/2026
0.1.12	103	2/26/2026
0.1.11	105	2/21/2026
0.1.10	105	2/20/2026
0.1.7	122	2/20/2026
0.1.6	119	1/9/2026
0.1.5	118	1/9/2026
0.1.4	116	12/28/2025
0.1.3	118	12/28/2025
0.1.2	156	12/26/2025
0.1.1	158	12/26/2025
0.1.0	179	12/25/2025