Marshal 24.11.15

Marshal API Tutorial

The Marshal package contains the engine (marshal.dll) for exporting, and importing, data objects using Marshal ETL models. Design your model in Marshal Studio, then use this package to execute it, as part of your own application.

Once the package is in your Visual Studio solution, you may use the Marshal.API namespace. There are other namespaces in the assembly, refrain from using them, since they expose higher complexity, and could be subject to future changes.

using Marshal.API;

In order to run an export, you need an instance of the Marshal.API.Model class, wrapping your Marshal model. If your export is large, which Marshal exports usually are, it is advisable to register a callback function, so that you may report progress to the user, or operator. Then invoke the Export method, by passing it an existing directory, and the name of a non-existent subdirectory. Summing this up, we recommend something similar to the following:

try
{
  Model model = new Model(@"C:\models\my-model.marshal")
  {
    ExportCallback = (exi) => Console.WriteLine("export {0}: {1}", exi.Status, exi.Message)
  };
	
  model.Export(new DirectoryInfo(@"C:\exports"), "MyExport");
}
catch (Model.LoadException ex)
{
  Console.Error.WriteLine("Failed to load model: {0}", ex.Message);
}
catch (Model.ExportException ex)
{
  Console.Error.WriteLine("Export error: {0}", ex.Message);
}

If successful, the above example will result in the folder C:\exports\MyExport, containing any and all content exported. Actually, the Export method returns an object implementing Model.IExport, that could aid in the further use, or processing of the export. Suppose, for instance, you would like to iterate over the exported data objects, do something with them, and then delete the export.

  Model.IExport export = model.Export(new DirectoryInfo(@"C:\exports"), "MyExport");
  export.ForEach(fileInfo => DoMyStuff(fileInfo));
  export.Cleanup();

Note, Cleanup may throw an exception, Model.CleanupException, so you should extend the catching as well. Also, if the model corresponds to a scalar function, returning a single data object, with no externally stored files, you could run the export in memory, to avoid file system storage altogether.

  Model.IMemExport export = model.Export();

A Marshal model may require parameters to be passed to it at runtime. It is possible to pass a single parameter when running Export, as well as to set model-wide named parameters when loading the model. To pass a parameter at export, typically used for filtering purposes by the model top-level function, just add it as a third argument.

  model.Export(new DirectoryInfo(@"C:\exports"), "MyExport", "CaseNumber LIKE 'KS 2017%'");

Setting model-wide parameters takes a bit more coding. They are used to pass configuration to a model, designed for reuse. When using model-wide parameters, your code needs to retrieve the parameters available, then set them, and pass them to the Model constructor.

  string modelPath = @"C:\models\my-model.marshal";
  Model.IParameterSet parameterSet = Model.DefinedParameters(modelPath);
  Model.IParameter myParameter = parameterSet.GetParameter("$EXCEL-FILE");
  if (myParameter != null)
  {
    myParameter.Value = @"C:\datasources\mydata.xlsx";
  }
  Model model = new Model(modelPath, parameterSet);

A Marshal model may contain custom functions, i.e source code (c#) that is part of the model, and compiled when the model is loaded. The Marshal engine will not compile the code, unless it is signed for use on the computer, where it is to run. Signing is normally done using Marshal Studio, but in a controlled environment your application could sign, on behalf of the user running the application.

  if (model.CustomCode.Exists && !model.CustomCode.IsOk)
  {
    model.CustomCode.Sign(); // actually, sign and recompile
  }     
  model.Export(new DirectoryInfo(@"C:\exports"), "MyExport");

To apply any model import rules, on a new or previously created export, call the Import method of the returned export object, implementing Model.IExport. If you are importing a previously generated export, use the model Open method, to get an export object, valid for the model. Importing, too, could take a while, so it is a good idea to register a callback function, that reports some kind of progress to the user.

try
{
  Model model = new Model(@"C:\models\my-model.marshal")
  {
    ImportCallback = (imi) => Console.WriteLine("import {0}: {1}", imi.Status, imi.Message)
  };
  
  model
    .Open(new FileInfo(@"C:\exports\MyExport\MyExport_index.xml"))
    .Import();
}
catch (Model.LoadException ex)
{
  Console.Error.WriteLine("Failed to load model: {0}", ex.Message);
}
catch (Model.OpenException ex)
{
  Console.Error.WriteLine("Failed to open export: {0}", ex.Message);
} 
catch (Model.ImportException ex)
{
  Console.Error.WriteLine("Import error: {0}", ex.Message);
}  

Note, that you need to pass a file to Open, not the enclosing directory. For table-valued models, resulting in multiple data objects, this should be the delivery note, stored at the root of the export. If the model is a scalar function, resulting in a single data object, this should be the data object itself, i.e C:\exports\MyExport\MyExport.xml in the above example.