OfficeIMO.Reader 2.0.1

OfficeIMO.Reader - document extraction facade

OfficeIMO.Reader is a read-only facade for deterministic document extraction. It normalizes supported source files into ReaderChunk objects for search, indexing, chat, RAG, migration, and review workflows.

Install

dotnet add package OfficeIMO.Reader

Quick start

using OfficeIMO.Reader;

foreach (var chunk in DocumentReader.Read(@"C:\Docs\Policy.docx")) {
    Console.WriteLine(chunk.Id);
    Console.WriteLine(chunk.Location.HeadingPath);
    Console.WriteLine(chunk.Markdown ?? chunk.Text);
}

Streams and folders

using OfficeIMO.Reader;

using var stream = File.OpenRead(@"C:\Docs\Policy.docx");
var chunksFromStream = DocumentReader.Read(stream, "Policy.docx").ToList();

var folderChunks = DocumentReader.ReadFolder(
    folderPath: @"C:\Docs",
    folderOptions: new ReaderFolderOptions {
        Recurse = true,
        MaxFiles = 500,
        MaxTotalBytes = 500L * 1024 * 1024,
        SkipReparsePoints = true,
        DeterministicOrder = true
    },
    options: new ReaderOptions {
        MaxChars = 8_000
    }).ToList();

What it reads

Built-in and modular adapters can extract:

• Word (.docx, .docm, .doc) as Markdown chunks.
• Excel (.xlsx, .xlsm, .xls) as table chunks and optional Markdown previews.
• PowerPoint (.pptx, .pptm) as slide-aligned chunks, optionally including notes.
• Markdown (.md, .markdown) as parser-aware heading chunks.
• EML/MIME, Outlook MSG/MAPI, TNEF/winmail.dat, and mbox as message, body, attachment, and embedded-item chunks. Rich results include attachment assets and typed Outlook metadata.
• OpenDocument (.odt, .ods, .odp), PDF, RTF, Visio, HTML, CSV/TSV, JSON, XML, YAML, EPUB, ZIP, and structured text through modular adapter packages.

Modular adapters

For services and concurrent hosts, build an isolated reader with only the adapters you need:

using OfficeIMO.Reader.Csv;
using OfficeIMO.Reader.Epub;
using OfficeIMO.Reader.Html;
using OfficeIMO.Reader.Json;
using OfficeIMO.Reader.OpenDocument;
using OfficeIMO.Reader.Pdf;
using OfficeIMO.Reader.Rtf;
using OfficeIMO.Reader.Visio;
using OfficeIMO.Reader.Xml;
using OfficeIMO.Reader.Yaml;
using OfficeIMO.Reader.Zip;

OfficeDocumentReader reader = new OfficeDocumentReaderBuilder()
    .AddCsvHandler()
    .AddEpubHandler()
    .AddHtmlHandler()
    .AddJsonHandler()
    .AddOpenDocumentHandler()
    .AddPdfHandler()
    .AddRtfHandler()
    .AddVisioHandler()
    .AddXmlHandler()
    .AddYamlHandler()
    .AddZipHandler()
    .Build();

var chunks = reader.Read(@"C:\Docs\data.json").ToList();

Build() freezes the handler configuration. The resulting OfficeDocumentReader is safe to reuse across concurrent reads and is unaffected by later builder changes. DocumentReader remains the simple built-in-only facade; modular formats are configured explicitly on OfficeDocumentReaderBuilder, so one reader cannot change another reader or process-wide state.

Async and bounded batches

Configure the instance-wide async limit once, then use the same reader for individual or batched work:

using OfficeIMO.Reader;

OfficeDocumentReader reader = new OfficeDocumentReaderBuilder()
    .WithMaxConcurrentReads(4)
    .Build();

OfficeDocumentReadResult document = await reader.ReadDocumentAsync(
    @"C:\Docs\Policy.docx",
    cancellationToken: cancellationToken);

IReadOnlyList<OfficeDocumentReadResult> documents = await reader.ReadDocumentsAsync(
    Directory.EnumerateFiles(@"C:\Docs", "*.docx"),
    batchOptions: new ReaderBatchOptions {
        MaxDegreeOfParallelism = 3,
        MaxDocuments = 500
    },
    cancellationToken: cancellationToken);

ReadDocumentsAsync(...) starts no more than the configured degree of parallelism, rejects input beyond MaxDocuments, cancels sibling workers after a failure, and returns results in the same order as the input paths. Handlers can provide ReadDocumentPathAsync or ReadDocumentStreamAsync for native asynchronous work. Existing synchronous format engines remain compatible and are scheduled through the instance's bounded worker gate.

Ordered document processors

Processors are opt-in transformations over OfficeDocumentReadResult. The builder freezes their order with the handler configuration, and the reader applies them to Read(...), ReadAsync(...), rich document reads, JSON reads, structured reads, and ReadDocumentsAsync(...):

OfficeDocumentReader reader = new OfficeDocumentReaderBuilder()
    .AddProcessor(new OfficeDocumentBlockNormalizationProcessor())
    .AddProcessor(new OfficeDocumentTableNormalizationProcessor())
    .AddProcessor(new OfficeDocumentLinkNormalizationProcessor())
    .AddProcessor(new OfficeDocumentArtifactClassificationProcessor())
    .AddProcessor(new OfficeDocumentAssetFilterProcessor(
        asset => asset.LengthBytes <= 5 * 1024 * 1024))
    .WithProcessorFailureBehavior(
        OfficeDocumentProcessorFailureBehavior.ContinueWithDiagnostic)
    .Build();

OfficeDocumentReadResult document = reader.ReadDocument(@"C:\Docs\Policy.docx");

The built-ins normalize shared blocks, list and heading levels, tables, and links; classify repeated page-boundary text; and filter assets together with dependent OCR candidates. They do not run unless registered, so an unconfigured reader preserves existing output. Add host behavior with OfficeDocumentProcessorBase, IOfficeDocumentProcessor, or DelegateOfficeDocumentProcessor.

The default failure policy is Throw. ContinueWithDiagnostic records processor-failed and runs later steps, while StopWithDiagnostic records the failure and marks later steps as skipped. Call ProcessDocument(...) or ProcessDocumentAsync(...) directly when the host needs the per-step OfficeDocumentProcessingResult. Processor instances attached to a shared reader must be safe for concurrent calls and should avoid retaining per-document state.

Folder enumeration remains a chunk/source traversal surface and does not run rich-document processors automatically. Use ReadDocumentsAsync(...) when every file must pass through the configured document pipeline.

Bounded structured extraction

Structured extraction projects the shared document model into deterministic scalar records, heading sections, named tables, typed forms, and diagnostics without adding an AI client or format-specific dependency:

OfficeDocumentStructuredExtractionResult extracted = reader.ReadStructured(
    @"C:\Docs\Policy.docx",
    structuredOptions: new OfficeDocumentStructuredExtractionOptions {
        MaxRecords = 5_000,
        MaxSections = 500,
        MaxSectionCharacters = 50_000,
        MaxTables = 250,
        MaxForms = 1_000,
        MaxDiagnostics = 500
    });

foreach (OfficeDocumentStructuredRecord record in extracted.Records) {
    Console.WriteLine($"{record.Category}: {record.Name} = {record.Value}");
}

string json = extracted.ToJson(indented: true);

The extractor recognizes metadata, forms, two-column key/value tables, Path/Type/Value tables, Visio Shape Data, chart summaries, table and visual quality, chunk readiness, and security/OCR/limit diagnostics. Each collection and section body has an explicit positive limit; reaching a bound adds a structured limit diagnostic instead of silently growing output. The non-generic schema-friendly shape is versioned independently from the stable OfficeDocumentReadResult transport. It is currently a serialization and host-integration contract; a generic ExtractStructured<T>() mapper and model-assisted extraction are intentionally outside the core package.

Token-aware hierarchical chunks

ReadHierarchical(...) keeps ReaderChunk as the embedding leaf while adding document, page/slide/sheet, and heading nodes around it:

ReaderChunkHierarchyResult hierarchy = reader.ReadHierarchical(
    @"C:\Docs\Policy.docx",
    chunkingOptions: new ReaderHierarchicalChunkingOptions {
        MaxTokens = 800,
        OverlapTokens = 80,
        MaxInputChunks = 10_000,
        MaxOutputChunks = 50_000,
        MaxHierarchyDepth = 32,
        MaxContextCharacters = 4_096,
        IncludeContextInText = true
    });

foreach (ReaderChunk chunk in hierarchy.Chunks) {
    StoreEmbedding(chunk.Id, chunk.Text, chunk.TokenEstimate ?? 0);
}

string sidecarJson = hierarchy.ToJson(indented: true);

The result records the selected token counter, original/output/overlap/context token totals, exact source character spans, deterministic leaf IDs and hashes, and a flattened hierarchy with explicit parent/child IDs. Splits prefer paragraph, line, sentence, and whitespace boundaries but enforce the token maximum even when a source block is larger. Structured tables, visuals, forms, actions, diagnostics, and warnings stay on the first segment so overlap does not duplicate sidecars.

The dependency-free default uses the existing four-characters-per-token heuristic. Supply a deterministic, thread-safe IReaderTokenCounter when the embedding model needs its exact tokenizer. A hierarchy represents one source document and rejects mixed-source chunk collections. Context and input/output/depth bounds emit structured diagnostics; invalid token budgets or counters fail explicitly.

This is an opt-in projection with its own schema id/version. It does not add fields to ReaderChunk, change normal reads, or alter the stable OfficeDocumentReadResult v5 transport. Static and instance sync/async file, stream, and byte overloads are available; instance reads apply configured processors before chunking.

Stable document transport

OfficeDocumentReadResult schema version 5 is the first stable JSON transport contract. Versions 1 through 4 were experimental and are deliberately rejected when reading transport payloads.

OfficeDocumentReadResult document = reader.ReadDocument(@"C:\Docs\Policy.docx");
string json = OfficeDocumentReadResultJson.Serialize(document, indented: true);

OfficeDocumentReadResult restored = OfficeDocumentReadResultJson.Deserialize(json);
Console.WriteLine(restored.SchemaVersion); // 5

string jsonSchema = OfficeDocumentReadResultSchema.GetJsonSchema();

The package embeds the versioned JSON Schema and also ships it as schemas/officeimo.document.read-result.v5.schema.json. Deserialization accepts only the current schema id/version, rejects unknown top-level members and incomplete envelopes, and returns a typed OfficeDocumentReadResultSchemaException for incompatible versions. Published core and modular Reader packages run NuGet package validation against their current public versions, so accidental API breaks fail packaging. A brand-new package opts out only until its first public baseline exists.

Content detection and structured diagnostics

Detect(...) and DetectAsync(...) report extension and content evidence without reducing the answer to one opaque enum:

ReaderDetectionResult detection = reader.Detect(@"C:\Inbox\upload.bin");

Console.WriteLine($"Selected: {detection.Kind}");
Console.WriteLine($"Extension: {detection.ExtensionKind}");
Console.WriteLine($"Content: {detection.ContentKind} ({detection.ContentConfidence})");
Console.WriteLine(string.Join(", ", detection.Evidence));

Standalone detection prefers medium- or high-confidence content evidence. Normal reads use ContentWhenUnknown by default, which preserves known-extension behavior while identifying unknown uploads. Opt into mislabeled-file routing when needed:

OfficeDocumentReadResult result = reader.ReadDocument(
    @"C:\Inbox\actually-markdown.txt",
    new ReaderOptions { DetectionMode = ReaderDetectionMode.PreferContent });

Seekable stream probes restore the original position. Prefix probing is capped at 4 MiB, and ZIP-based Office/Visio/EPUB inspection walks at most 4,096 local entries without decompressing archive payloads. Detection mismatches, detected unknown inputs, parser failures, limits, truncation, unsupported content, read failures, and OCR readiness are exposed through OfficeDocumentDiagnostic with stable Category, Code, Source, IsRecoverable, and Attributes fields.

Rich built-in document mappings

Word, Excel, and PowerPoint rich reads use the format packages' public inspection models instead of reparsing Open XML in the Reader facade:

OfficeDocumentReadResult word = DocumentReader.ReadDocument("policy.docx");
OfficeDocumentReadResult workbook = DocumentReader.ReadDocument("forecast.xlsx");
OfficeDocumentReadResult deck = DocumentReader.ReadDocument("briefing.pptx");

Console.WriteLine($"{word.Blocks.Count} semantic Word blocks");
Console.WriteLine($"{workbook.Tables.Count} named Excel tables");
Console.WriteLine($"{deck.Visuals.Count} PowerPoint chart snapshots");

The Word mapping preserves headings, lists, links, tables, and header/footer content. Excel exposes worksheet locations, formal tables, cell links, formula/comment/named-range counts, and workbook properties. PowerPoint exposes slide-local blocks and tables, run and shape links, chart snapshots, and point-based geometry. Modular HTML, EPUB, RTF, and Visio adapters register the same native v5 result surface when their packages are installed.

Optional OCR execution

The core defines IOfficeOcrEngine and runs caller-supplied engines without taking an OCR or cloud SDK dependency. Execution resolves candidate assets, validates payload hashes and media types, and bounds candidate count, per-candidate and total bytes, concurrency, duration, recognized characters, and detailed spans:

var engine = new DelegateOfficeOcrEngine(
    "host-vision-service",
    async (request, cancellationToken) => {
        HostVisionResponse response = await visionClient.RecognizeAsync(
            request.Payload,
            request.Language,
            cancellationToken);

        return new OfficeOcrEngineResult {
            Text = response.Text,
            Confidence = response.Confidence,
            Language = response.Language,
            Provider = "host-vision-service"
        };
    });

OfficeDocumentReadResult source = DocumentReader.ReadDocument("scanned.pdf");
OfficeDocumentOcrExecutionResult execution = await source.ApplyOcrAsync(
    engine,
    new OfficeDocumentOcrExecutionOptions {
        MaxCandidates = 25,
        MaxDegreeOfParallelism = 2
    });

To run the same frozen configuration automatically for every rich read, register new OfficeDocumentOcrProcessor(engine, executionOptions) with OfficeDocumentReaderBuilder.AddProcessor(...). Engines that do not advertise concurrent-request support are serialized per engine instance even when several reader operations run concurrently.

execution.Document contains merged ocr-text blocks/chunks while unresolved candidates and diagnostics remain intact. execution.Recognitions carries optional line, word, and character spans with confidence, language, bounding boxes, and coordinate units. Detailed provider spans intentionally stay outside the stable v5 transport; the merged text and trace metadata use the existing schema.

Use OfficeIMO.Reader.Ocr.Process for a versioned local executable/service bridge or OfficeIMO.Reader.Ocr.Tesseract for an installed Tesseract CLI. Neither package is pulled transitively by OfficeIMO.Reader.

Host examples

Capability discovery

using OfficeIMO.Reader;

var reader = new OfficeDocumentReaderBuilder().Build();
var capabilities = reader.GetCapabilities();
foreach (var capability in capabilities) {
    Console.WriteLine($"{capability.Id}: {string.Join(", ", capability.Extensions)}");
}

string manifestJson = reader.GetCapabilityManifestJson();

Register a custom handler

using OfficeIMO.Reader;

OfficeDocumentReader reader = new OfficeDocumentReaderBuilder()
    .AddHandler(new ReaderHandlerRegistration {
        Id = "custom-audit",
        DisplayName = "Custom audit reader",
        Kind = ReaderInputKind.Text,
        Extensions = new[] { ".auditx" },
        ReadPath = (path, options, cancellationToken) => {
            string text = File.ReadAllText(path);
            return new[] {
                new ReaderChunk {
                    Id = "audit:1",
                    Kind = ReaderInputKind.Text,
                    Text = text,
                    Location = new ReaderLocation { Path = path }
                }
            };
        }
    })
    .Build();

Handlers that already expose a structured document model can register rich result delegates instead of rebuilding that model as chunks first:

OfficeDocumentReader reader = new OfficeDocumentReaderBuilder()
    .AddHandler(new ReaderHandlerRegistration {
        Id = "custom-rich-reader",
        DisplayName = "Custom rich reader",
        Kind = ReaderInputKind.Text,
        Extensions = new[] { ".rich" },
        ReadDocumentPath = (path, options, cancellationToken) => ReadRichDocument(path),
        ReadDocumentStream = (stream, sourceName, options, cancellationToken) => ReadRichDocument(stream, sourceName)
    })
    .Build();

reader.ReadDocument(...) dispatches directly to these delegates. Existing reader.Read(...) calls remain usable by projecting the returned result's Chunks collection. A handler may continue to register ReadPath and ReadStream when chunk production is its native contract.

Adapter authors can call DocumentReader.CreateDocumentResult(...) to create the common v5 source, chunks, assets, diagnostics, and baseline collections, then replace or enrich format-owned blocks, pages, tables, links, forms, visuals, and metadata.

Host contracts

• ReaderOptions controls chunk size, table row limits, footnotes/notes, Excel ranges, Markdown heading chunking, hashes, and input budgets.
• ReaderFolderOptions controls recursion, file limits, byte limits, reparse-point handling, and deterministic folder order.
• OfficeDocumentReader.GetCapabilities() and GetCapabilityManifestJson() expose the frozen configuration of that reader instance.
• Capability records distinguish basic path/stream support from native rich-result support through SupportsDocumentPath and SupportsDocumentStream.
• SupportsAsyncPath and SupportsAsyncStream identify handlers with native asynchronous delegates; false means the async facade uses the bounded synchronous fallback.
• DocumentReader.Detect(...) / DetectAsync(...) and OfficeDocumentReader.Detect(...) / DetectAsync(...) expose bounded extension/content evidence, confidence, media type, and mismatch state.
• OfficeDocumentDiagnostic carries stable categories, codes, sources, recoverability, and attributes so hosts do not need to parse warning text.
• OfficeDocumentReadResultJson reads and writes stable schema version 5 envelopes; OfficeDocumentReadResultSchema.GetJsonSchema() exposes the packaged schema artifact.
• OfficeDocumentProcessorPipeline freezes ordered processors and reports each completed, failed, or skipped step.
• OfficeDocumentStructuredExtractor produces bounded non-generic records, sections, named tables, forms, and diagnostics without AI dependencies.
• ReaderHierarchicalChunker produces token-bounded ReaderChunk leaves, exact overlap spans, and document/container/heading hierarchy nodes.
• OfficeDocumentReaderBuilder.AddHandler(...) is the recommended custom-handler path for services and concurrent hosts.
• Static DocumentReader registration is retained as a process-wide compatibility surface.

Boundaries

• OfficeIMO.Reader owns the shared extraction contract and built-in facade.
• Source-specific parsing belongs in the source package or modular adapter.
• Email RTF transport decoding belongs to OfficeIMO.Email; semantic extraction of an RTF-only email body is delegated to the registered OfficeIMO.Reader.Rtf adapter.
• Adapters should use ReaderInputLimits so input size and stream behavior stays consistent.
• AI or database storage belongs in the consuming application.

Performance evidence

OfficeIMO.Reader.Benchmarks covers rich extraction across all built-in and modular formats, bounded content detection, schema serialization/deserialization, and parser-versus-Reader isolation. Run it from the repository root with:

dotnet run --project OfficeIMO.Reader.Benchmarks/OfficeIMO.Reader.Benchmarks.csproj -c Release -f net8.0

The benchmark corpus is generated deterministically before measurement. Benchmark artifacts are machine-specific and remain untracked; durable baseline notes record the runtime, hardware, corpus, and relevant comparisons.

Related packages

• OfficeIMO.Email
• OfficeIMO.Reader.Pdf
• OfficeIMO.Reader.Rtf
• OfficeIMO.Reader.Visio
• OfficeIMO.Reader.Html
• OfficeIMO.Reader.Csv
• OfficeIMO.Reader.Ocr.Process
• OfficeIMO.Reader.Ocr.Tesseract
• OfficeIMO.Reader.Json
• OfficeIMO.Reader.OpenDocument
• OfficeIMO.Reader.Xml
• OfficeIMO.Reader.Yaml
• OfficeIMO.Reader.Epub
• OfficeIMO.Reader.Zip

Targets and license

• Targets: netstandard2.0, net8.0, net10.0.
• License: MIT.
• Repository: EvotecIT/OfficeIMO

Dependency footprint

• External: System.Text.Json for schema/result serialization.
• OfficeIMO: Native Word, Excel, PowerPoint, email, Markdown, PDF, and Drawing engines are reused directly; optional formats stay in adapter packages.

See the complete OfficeIMO package map for related formats and conversion paths.