Flowthru 0.1.11

There is a newer version of this package available.
See the version list below for details.

dotnet add package Flowthru --version 0.1.11

NuGet\Install-Package Flowthru -Version 0.1.11

This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.

<PackageReference Include="Flowthru" Version="0.1.11" />

For projects that support PackageReference, copy this XML node into the project file to reference the package.

<PackageVersion Include="Flowthru" Version="0.1.11" />
                    

                            Directory.Packages.props

<PackageReference Include="Flowthru" />
                    

                            Project file

For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.

paket add Flowthru --version 0.1.11

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

#r "nuget: Flowthru, 0.1.11"

#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.

#:package Flowthru@0.1.11

#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.

#addin nuget:?package=Flowthru&version=0.1.11
                    

                            Install as a Cake Addin

#tool nuget:?package=Flowthru&version=0.1.11
                    

                            Install as a Cake Tool

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

Flowthru

A type-safe data engineering framework for .NET inspired by Kedro's declarative pipelines.

Version: 0.1.11 (Alpha - Simplified AddNode API)
Status: Active Development

Quick Start

# Install
dotnet add package Flowthru

# Create your catalog
public class MyCatalog : DataCatalogBase
{
    public ICatalogEntry<IEnumerable<RawData>> RawData { get; }
    public ICatalogEntry<IEnumerable<ProcessedData>> ProcessedData { get; }
    
    public MyCatalog(string basePath)
    {
        _basePath = basePath;
        InitializeCatalogProperties();
    }
    
    private string _basePath;
    
    public ICatalogEntry<IEnumerable<RawData>> RawData =>
        GetOrCreateEntry(() => CatalogEntries.Enumerable.Csv<RawData>(
            label: "RawData", 
            filePath: $"{_basePath}/raw.csv"));
}

// Run your application
public class Program
{
    public static async Task<int> Main(string[] args)
    {
        var cli = FlowthruCliBuilder
            .Create(flowthru =>
            {
                flowthru.UseConfiguration();
                flowthru.UseCatalog(_ => new MyCatalog("Data"));
                
                // Register pipelines with descriptions
                flowthru
                    .RegisterPipeline<MyCatalog>("data_processing", DataProcessingPipeline.Create)
                    .WithDescription("Process raw data")
                    .WithTags("etl");
            })
            .ConfigureLogging(logging => logging.AddConsole())
            .Build();
            
        return await cli.RunAsync(args);
    }
}

# Execute
dotnet run data_processing

Documentation

Tutorials

TODO: Improve View Spaceflights test pipeline

Explanation

Understanding-oriented: Why Flowthru works the way it does.

Why Compile-Time Safety Matters
Understanding the Three Compile-Time Safety Layers
Common Pitfalls and How to Avoid Them
Design Rationale: Why These Choices?
Comparing with Kedro: Key Differences
When to Use Flowthru vs Kedro
Future Enhancements

Core Features

✅ Compile-Time Type Safety - Catch errors before execution, not during
✅ Configuration File Support - JSON/YAML configs with environment layering
✅ Eager Schema Validation - Validate external data before pipeline runs (optional)
✅ Read/Write Capability Enforcement - Prevent writes to read-only sources (Excel, APIs)
✅ Generic Pipeline Registry - Type-safe pipeline registration with parameters
✅ Property-Based Catalogs - IntelliSense-driven dataset access
✅ Expression-Based Mapping - Refactoring-safe multi-input/output nodes
✅ Application Builder Pattern - Minimal boilerplate (~15 lines)
✅ Formatted Console Output - Beautiful execution logs with progress tracking
✅ Dependency Injection - Property injection for services and loggers

Philosophy

Fail at compile-time, not runtime.

Every design decision in Flowthru prioritizes catching errors during compilation:

Typos in catalog keys → Compile error (property doesn't exist)
Wrong data types → Compile error (generic constraint violation)
Missing parameters → Compile error (property not found)
Type mismatches → Compile error (incompatible types)

The C# compiler becomes your documentation, showing all available datasets, parameters, and types through IntelliSense.

Comparison with Kedro

Aspect	Kedro	Flowthru
Type Safety	Runtime (duck typing)	Compile-time (generics)
Catalog	YAML + string keys	C# properties
Nodes	Functions with decorators	Classes inheriting `NodeBase<TIn,TOut>`
Parameters	YAML dictionaries	Strongly-typed record classes
Error Detection	During execution	During compilation
IDE Support	Basic	Full (IntelliSense, refactoring, navigation)

Requirements

.NET 9.0 or later
C# 12 or later

License

MIT License - See LICENSE file for details

Acknowledgments

Kedro: Inspiration for declarative pipeline patterns
Microsoft.Extensions: DI and logging infrastructure

Product	Compatible and additional computed target framework versions.
.NET	net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed.

Compatible target framework(s)

Included target framework(s) (in package)

Learn more about Target Frameworks and .NET Standard.

net10.0
- CsvHelper (>= 30.0.1)
- ExcelDataReader (>= 3.6.0)
- ExcelDataReader.DataSet (>= 3.6.0)
- Microsoft.Extensions.Configuration (>= 9.0.10)
- Microsoft.Extensions.Configuration.Binder (>= 9.0.10)
- Microsoft.Extensions.Configuration.Json (>= 9.0.10)
- Microsoft.Extensions.DependencyInjection (>= 9.0.10)
- Microsoft.Extensions.DependencyInjection.Abstractions (>= 9.0.10)
- Microsoft.Extensions.Logging (>= 9.0.10)
- Microsoft.Extensions.Logging.Abstractions (>= 9.0.10)
- Microsoft.Extensions.Logging.Console (>= 9.0.10)
- NetEscapades.Configuration.Yaml (>= 3.1.0)
- Parquet.Net (>= 5.2.0)
- System.ComponentModel.Annotations (>= 5.0.0)

NuGet packages (6)

Showing the top 5 NuGet packages that depend on Flowthru:

Package	Downloads
Flowthru.Integrations.MLNet ML.NET integration for Flowthru - provides IDataView container adapters for machine learning pipelines	1.0K
Flowthru.Extensions.ML Pure, unoptimized UMAP implementation for Flowthru - direct port from Python reference implementation by Leland McInnes	1.0K
Flowthru.Misc.ML Pure, unoptimized UMAP implementation for Flowthru - direct port from Python reference implementation by Leland McInnes	34
Flowthru.Extensions.EFCore Entity Framework Core integration for Flowthru data catalogs	29
Flowthru.Extensions.MLNet ML.NET integration for Flowthru - provides IDataView container adapters for machine learning pipelines	29

GitHub repositories

This package is not used by any popular GitHub repositories.

Version	Downloads	Last Updated
0.1.24	40	3/9/2026
0.1.23	45	3/9/2026
0.1.22	61	3/6/2026
0.1.16	90	3/5/2026
0.1.15	151	3/2/2026
0.1.14	109	2/28/2026
0.1.12	103	2/26/2026
0.1.11	105	2/21/2026
0.1.10	105	2/20/2026
0.1.7	119	2/20/2026
0.1.6	118	1/9/2026
0.1.5	117	1/9/2026
0.1.4	115	12/28/2025
0.1.3	117	12/28/2025
0.1.2	155	12/26/2025
0.1.1	157	12/26/2025
0.1.0	177	12/25/2025