BIT.Data.Sync.AspNetCore
1.0.12
See the version list below for details.
dotnet add package BIT.Data.Sync.AspNetCore --version 1.0.12
NuGet\Install-Package BIT.Data.Sync.AspNetCore -Version 1.0.12
<PackageReference Include="BIT.Data.Sync.AspNetCore" Version="1.0.12" />
paket add BIT.Data.Sync.AspNetCore --version 1.0.12
#r "nuget: BIT.Data.Sync.AspNetCore, 1.0.12"
// Install BIT.Data.Sync.AspNetCore as a Cake Addin #addin nuget:?package=BIT.Data.Sync.AspNetCore&version=1.0.12 // Install BIT.Data.Sync.AspNetCore as a Cake Tool #tool nuget:?package=BIT.Data.Sync.AspNetCore&version=1.0.12
SyncFramework
is c# a library that helps you synchronize any type of data, using delta encoding technology
There are currently 2 implementations of the SyncFramework
- SyncFramework for XPO
- SyncFramework for Entity Framework Core
in a nutshell
Data synchronization in EFCore is accomplished by replacing the DbContext's internal service collection with a custom one, containing the SyncFramework services.
The SyncFramework services are registered in the service collection using the extension method, AddEfSynchronization.
SyncFramework replaces the BatchExecutor service with a custom one, capable of intercepting the generation of SQL commands (Deltas), and storing them so they can be executed in a remote database.
Deltas are typically generated for the same database engine in use, but they can also be generated for a different database. For instance, your main database could be MS SQL Server, but your remote database could be PostgreSQL.
If you want to learn more about data synchronization you can checkout the following blog posts:
- Data synchronization in a few words - https://www.jocheojeda.com/2021/10/10/data-synchronization-in-a-few-words/
- Parts of a Synchronization Framework - https://www.jocheojeda.com/2021/10/10/parts-of-a-synchronization-framework/
- Let�s write a Synchronization Framework in C# - https://www.jocheojeda.com/2021/10/11/lets-write-a-synchronization-framework-in-c/
- Synchronization Framework Base Classes - https://www.jocheojeda.com/2021/10/12/synchronization-framework-base-classes/
- Planning the first implementation - https://www.jocheojeda.com/2021/10/12/planning-the-first-implementation/
- Testing the first implementation - https://youtu.be/l2-yPlExSrg
- Adding network support - https://www.jocheojeda.com/2021/10/17/syncframework-adding-network-support/
Current version 7.0.X
Target Framework net6.0
- EfCore Version 7.0.3
- Postgres Version 7.0.3
- Pomelo Mysql Version 7.0.0
- Sqlite Version 7.0.3
- Sql Server Version 7.0.3
Nugets
- Common base classes and interfaces to implement delta synchronization https://www.nuget.org/packages/BIT.Data.Sync/
- EfCore Implementation https://www.nuget.org/packages/BIT.Data.Sync.EfCore/
- EfCore Implementation for postgres https://www.nuget.org/packages/BIT.Data.Sync.EfCore.Npgsql/
- EfCore Implementation for Mysql/MariaDB using Pomelo provider https://www.nuget.org/packages/BIT.Data.Sync.EfCore.Pomelo.MySql/
- EfCore Implementation for Sqlite https://www.nuget.org/packages/BIT.Data.Sync.EfCore.Sqlite/
- EfCore Implementation for MsSqlServer https://www.nuget.org/packages/BIT.Data.Sync.EfCore.SqlServer/
Playground
Chcke our new playground demo here
https://syncframework.jocheojeda.com/
Getting started
The first step is to install the Nuget package that matches your current database, for example if you are using MsSqlServer you should install the following Nuget package
BIT.Data.Sync.EfCore.SqlServer
If you want to use the SyncFramework with a different database engine, you should install the corresponding Nuget package.
Let's prepare the services needed for database synchronization
Service collection
//The options needed for the DbContext, you can build them in any way you want, in this case we are using Sqlite
DbContextOptionsBuilder OptionsBuilder = new DbContextOptionsBuilder();
OptionsBuilder.UseSqlite(ConnectionString);
//The http client is used to send the deltas to the server
HttpClient Client = new HttpClient();
Client.BaseAddress = new Uri("https://ReplaceWithYourServerUrl/");
//The DeltaGenerator is used to generate the deltas, you can add as many as you want, in this case we are using the Sqlite and SqlServer generators
List<DeltaGeneratorBase> DeltaGenerators = new List<DeltaGeneratorBase>();
DeltaGenerators.Add(new SqliteDeltaGenerator());
DeltaGenerators.Add(new SqlServerDeltaGenerator());
DeltaGeneratorBase[] additionalDeltaGenerators = DeltaGenerators.ToArray();
//We prepare the service collection that will be used to register the SyncFramework services
ServiceCollection ServiceCollection = new ServiceCollection();
ServiceCollection.AddEfSynchronization((options) =>
{
string ConnectionString $"Data Source=Deltas.db;";
//The EfSynchronizationOptions is used to configure the database synchronization
options.UseSqlite(ConnectionString);// we are going to store the deltas in a Sqlite database
//it's possible to store deltas any any of the supported database engines, for example you could use MsSqlServer
//options.UseSqlServer(ConnectionString);
},
//The http client that will send the data to the server
Client,
//the ID of the delta store on the server side,a sync server can have multiple delta stores
"MemoryDeltaStore1",
//the ID of this client, each client has to have an unique id
Maui",
//Additional delta generators, in this case we are using the Sqlite and SqlServer generators
additionalDeltaGenerators);
//We add the entity framework services, in this case we are using Sqlite
ServiceCollection.AddEntityFrameworkSqlite();
//We build the service provider
var ServiceProvider = ServiceCollection.BuildServiceProvider();
The DbContext
You should use SyncFrameworkDbContext instead of the regular DbContext. SyncFrameworkDbContext is a subclass of the regular DbContext. The only difference is that it implements all the boilerplate code needed for synchronization, so you can use it in the same way.
MyDbContext MyDbContext = new MyDbContext(OptionsBuilder.Options, ServiceProvider);
MyDbContext.Database.EnsureCreated();
If you don't want to use SyncFrameworkDbContext you can extend your own DbContext by implementing ISyncClientNode interface.
public class MyAppDbContext : DbContext, ISyncClientNode
Then, you need to create a new instance of the delta processor and the services provider that we built in previous steps. Let's do that in the constructor of your DbContext.
public MyAppDbContext(DbContextOptions options,IServiceProvider serviceProvider) : base(options)
{
this.serviceProvider = serviceProvider;
this.DeltaProcessor = new EFDeltaProcessor(this);
}
Then we need to replaces the service provider with our own, let's do that by override or extend the configuration method as follows
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
this.Identity = serviceProvider.GetService<ISyncIdentityService>()?.Identity;
this.DeltaStore = serviceProvider.GetService<IDeltaStore>();
this.SyncFrameworkClient = serviceProvider.GetService<ISyncFrameworkClient>();
optionsBuilder.UseInternalServiceProvider(serviceProvider);
}
Our DbContext is able to store and process deltas and push and pull deltas from a SyncServer.
We are done!!!
Fetching pulling and pushing deltas
The operations of fetch, pull and push are generic and they are implemented as extension methods of in the ISyncClientNode interface.
- Fetch:Fetches the deltas from the server and returns them as a list of Delta objects
- Pull: Fetches the deltas from the server, process them by execute the sql operations in your local database
- Push: Pushes the deltas to the server
await MyDbContext.PushAsync();
await MyDbContext.PullAsync();
await MyDbContext.FetchAsync();
Best practices using your DbConext to push and pull deltas
You should use your DbContext withing a using statement, this will ensure that the DbContext is disposed and the connection is closed, also that the http client is not dissposed (this can happend in platforms like MAUI).
using (var context = GetDbContext())
{
// use your conext here
}
Implementing a SyncServer
The easiest way to implement a server is to add register a instace of SyncServer in the service collection.
public void ConfigureServices(IServiceCollection services)
{
SyncServerNode syncServerNode = new SyncServerNode(new MemoryDeltaStore(), null, "MemoryDeltaStore1");
services.AddSingleton<ISyncServer>(new BIT.Data.Sync.Server.SyncServer(syncServerNode));
}
and use that service in your controller
public class SyncController : ControllerBase
{
private readonly ILogger<SyncController> _logger;
private readonly ISyncServer _SyncServer;
protected string GetHeader(string HeaderName)
{
Microsoft.Extensions.Primitives.StringValues stringValues = HttpContext.Request.Headers[HeaderName];
return stringValues;
}
public SyncController(ILogger<SyncController> logger, ISyncServer SyncServer)
{
_logger = logger;
_SyncServer = SyncServer;
}
[HttpPost(nameof(Push))]
public virtual async Task Push()
{
string NodeId = GetHeader("NodeId");
var stream = new StreamReader(this.Request.Body);
var body = await stream.ReadToEndAsync();
if (string.IsNullOrEmpty(body))
return;
using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(body)))
{
DataContractJsonSerializer deserializer = new DataContractJsonSerializer(typeof(List<Delta>));
List<Delta> Deltas = (List<Delta>)deserializer.ReadObject(ms);
await _SyncServer.SaveDeltasAsync(NodeId, Deltas, new CancellationToken());
var Message = $"Push to node:{NodeId}{Environment.NewLine}Deltas Received:{Deltas.Count}{Environment.NewLine}Identity:{Deltas.FirstOrDefault()?.Identity}";
_logger.LogInformation(Message);
Debug.WriteLine(Message);
}
}
[HttpGet("Fetch")]
public async Task<string> Fetch(Guid startindex, string identity = null)
{
string NodeId = GetHeader("NodeId");
var Message = $"Fetch from node:{NodeId}{Environment.NewLine}Start delta index:{startindex}{Environment.NewLine}Client identity:{identity}";
_logger.LogInformation(Message);
Debug.WriteLine(Message);
IEnumerable<IDelta> enumerable;
if (string.IsNullOrEmpty(identity))
enumerable = await _SyncServer.GetDeltasAsync(NodeId, startindex, new CancellationToken());
else
enumerable = await _SyncServer.GetDeltasFromOtherNodes(NodeId, startindex, identity, new CancellationToken());
List<Delta> toserialzie = new List<Delta>();
var knowTypes = new List<Type>() { typeof(DateTimeOffset) };
foreach (IDelta delta in enumerable)
{
toserialzie.Add(new Delta(delta));
}
DataContractJsonSerializer js = new DataContractJsonSerializer(typeof(List<Delta>), knowTypes);
MemoryStream msObj = new MemoryStream();
js.WriteObject(msObj, toserialzie);
msObj.Position = 0;
StreamReader sr = new StreamReader(msObj);
string jsonDeltas = sr.ReadToEnd();
return jsonDeltas;
}
}
Installing project templates
dotnet new -i BIT.Data.Sync.Templates
To create a new SyncServer
dotnet new SyncServer -o MySyncServer
Changelog
You can see the current list of changes here
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net6.0
- BIT.Data.Sync (>= 1.0.12)
- Microsoft.AspNetCore.Mvc.Core (>= 2.2.5)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
2.0.0-beta | 80 | 6/11/2024 |
1.0.13 | 163 | 5/30/2024 |
1.0.12 | 90 | 5/15/2024 |
1.0.2 | 669 | 7/4/2023 |