ETLBox.SQLite 3.6.0

ETLBox - Lightweight ETL and Data Integration for .NET

Get Started | Overview | Code Examples | Licensing | Free Trial

What is ETLBox?

ETLBox is a lightweight ETL (Extract, Transform, Load) library and data integration toolbox for .NET. It helps you create and manage data integration pipelines to extract data from different sources, transform it into the format you need, and load it into databases, APIs, or files. Whether you're migrating data, synchronizing systems, or automating workflows, ETLBox provides the components to handle these tasks efficiently.

ETLBox is a flexible alternative to traditional ETL tools like SQL Server Integration Services (SSIS) or Azure Data Factory. Unlike those tools, ETLBox does not rely on a graphical user interface, making it ideal for developers who prefer programmatic control. It supports parallel execution and real-time processing, making it a solid choice for building scalable data solutions.

Feature Overview

• Scalable Pipelines - Handle large data volumes with bulk operations and parallel processing.
• Flexible Connectors - Connect to SQL, NoSQL, APIs, and flat files like CSV, Excel, and JSON.
• Streaming Support - Process real-time data streams and keep workflows updated.
• Transformations - Harmonize, filter, aggregate, and clean data with built-in tools.
• Cloud-Ready - Deploy on Azure, AWS, Docker, and Kubernetes.
• Custom Extensions - Build and integrate your own transformations and connectors.

Why Choose ETLBox?

ETLBox lets you build data pipelines that process structured and unstructured data from various sources, including databases, APIs, and flat files. It automates tasks like data transformation, validation, and cleaning, and supports bulk inserts, updates, and deletes. With its ability to handle streaming data and parallel workflows, ETLBox delivers performance and flexibility for modern data integration tasks. Developers can rely on built-in connectors or extend the library to fit custom requirements.

Using ETLBox

Example 1: Bulk Insert into SQL Database

string connString =
  @"Source=.;Trusted_Connection=true;Initial Catalog=ETLBox;"
var destCon = new SqlConnectionManager(connString);

//Create the dataflow components
var source = new CsvSource(sourceCon, "SourceData.csv");
var dest = new DbDestination(destCon, "DestinationTable");

//Default mapping: Csv header and column name match (case-sensitive)
//Override mapping like this:
dest.ColumnMapping = new[] {
    new DbColumnMap() {
      DbColumnName = "Id", PropertyName = "OrderNumber"
    }
};

//Link the source and destination, run the dataflow
source.LinkTo(dest);
await Network.ExecuteAsnyc(source);

Example 2: Migrate Between Databases

string mySqlConnString =
  @"Server=localhost;Database=ETLBox;Uid=user;Pwd=password;";
string postgresConnString =
  @"Server=.;Initial Catalog=ETLBox;Trusted_Connection=true;";
var sourceCon = new MySqlConnectionManager(mySqlConnString);
var destCon = new PostgresConnectionManager(postgresConnString);

//Create the dataflow components
var source = new DbSource(sourceCon, "SourceTableName");
var dest = new DbDestination(destCon, "DestTableName");

//Link the source and destination, run the dataflow
source.LinkTo(dest);
await Network.ExecuteAsync(source);

Example 3: Merge Data Between Tables

public class MyMergeRow : MergeableRow
{
    [IdColumn]
    public long Key { get; set; }
    [CompareColumn]
    [UpdateColumn]
    public string Value { get; set; }
    [DeleteColumn(true)]
    public bool DeleteThisRow { get; set; }
}

//Create the dataflow components
DbSource<MyMergeRow> source = new (connection, "SourceTable");
DbMerge<MyMergeRow> mergeDest = new (connection, "DestinationTable");
merge.MergeMode = MergeMode.Full;
merge.CacheMode = CacheMode.Partial;

//Link the source and destination, run the dataflow
source.LinkTo(merge);
Network.Execute(source);

Licensing & Support

For our full list of code examples, tutorials, licensing information, and documentation visit: www.etlbox.net

For more support and inquiries, please email us at: support@etlbox.net

Documentation Links

• Get Started Guide
• Overview
• Code Examples
• Licensing
• Free Trial
• How-To Guides

Ready to build your ETL pipelines? Start now at www.etlbox.net.