Meziantou.Framework.Language.Xml 4.0.6

Meziantou.Framework.Language.Xml

An immutable XML syntax tree that keeps every character of the text it was parsed from — comments, whitespace, attribute quoting and all — and lets you edit it without reformatting anything you did not touch.

It is modelled on Roslyn. If you have used Microsoft.CodeAnalysis, everything here will look familiar: a SyntaxTree over a SourceText, nodes and tokens with spans, trivia, SyntaxKind, visitors, rewriters, and annotations. On top of that it speaks XPath, so a node can be found the way XML people expect to find one.

Parsing never throws and never gives up. Whatever the text says, the tree reproduces it exactly, and anything wrong with it is reported through GetDiagnostics().

The types are in the Meziantou.Framework.Language.Xml namespace, and the shared ones (SyntaxAnnotation, TextChange, …) in Meziantou.Framework.Language.

const string Xml = "<root><item version='1.0.0' /></root>";

var tree = XmlSyntaxTree.ParseText(Xml);

// The text comes back exactly as it went in.
Assert.Equal(tree.GetText().Text, tree.GetRoot().ToFullString());

// Find a node with XPath, and edit it without touching anything else.
var updated = tree.GetRoot().ReplaceNode("//item/@version", node => ((XmlAttributeSyntax)node).WithValue("2.0.0"));

Assert.Equal("<root><item version='2.0.0' /></root>", updated.ToFullString());

The shape of a tree

XmlDocumentSyntax
├── Nodes : SyntaxList<XmlNodeSyntax>
│   └── XmlElementSyntax
│       ├── StartTag : XmlElementStartTagSyntax  →  LessThanToken, NameToken, Attributes, GreaterThanToken
│       ├── Content  : SyntaxList<XmlNodeSyntax>
│       └── EndTag   : XmlElementEndTagSyntax?   →  LessThanSlashToken, NameToken, GreaterThanToken
└── EndOfFileToken

A node in a document or in an element's content is an XmlElementSyntax, XmlEmptyElementSyntax, XmlTextSyntax, XmlCommentSyntax, XmlCDataSectionSyntax, XmlDeclarationSyntax, XmlProcessingInstructionSyntax, XmlDocumentTypeSyntax, or XmlSkippedTextSyntax for text the parser could not use.

An element written as one self-closing tag is its own node type. <item/> is an XmlEmptyElementSyntax, which has attributes but no content and no end tag; <item></item> is an XmlElementSyntax. Both carry Name, Attributes, and GetAttribute(name).

Everything is a token, including the punctuation. <, >, />, =, and the quotes around an attribute value are each a token of their own, so an edit rebuilds the tree rather than splicing text:

var attribute = (XmlAttributeSyntax)tree.GetRoot().SelectSingleSyntaxNode("//item/@version")!;

Assert.Equal("version", attribute.NameToken.Text);
Assert.Equal("'", attribute.StartQuoteToken.Text);
Assert.Equal("1.0.0", attribute.ValueToken.Text);
Assert.Equal("1.0.0", attribute.Value);

Which whitespace is trivia

Inside a tag, whitespace is trivia; between tags it is content. That is what XML says: the space in <book id = '1'> carries no meaning, while the newline between two elements is character data an application may care about.

var tree = XmlSyntaxTree.ParseText("<root>\n  <book id = '1' />\n</root>");
var book = (XmlEmptyElementSyntax)tree.GetRoot().SelectSingleSyntaxNode("//book")!;

Assert.Equal(" ", book.Attributes[0].NameToken.LeadingTrivia.ToFullString());
Assert.Equal(" ", book.Attributes[0].EqualsToken.LeadingTrivia.ToFullString());

var root = (XmlElementSyntax)tree.GetRoot().Nodes[0];
Assert.Equal("\n  ", ((XmlTextSyntax)root.Content[0]).Text);

Editing without reformatting

An edit rebuilds only the path from the changed node up to the root. Everything else is carried over as-is, so the rest of the document is untouched — and the result is the same type you started with, so no cast is needed.

var tree = XmlSyntaxTree.ParseText("<root><a>1</a><b>2</b></root>");
var root = (XmlElementSyntax)tree.GetRoot().Nodes[0];
var b = root.Content.OfType<XmlElementSyntax>().Single(element => element.Name == "b");

XmlDocumentSyntax updated = tree.GetRoot().ReplaceNode(b, b.WithInnerText("two"));

Assert.Equal("<root><a>1</a><b>two</b></root>", updated.ToFullString());

// <a> was not touched, so it is the very same node.
Assert.True(root.Content[0].IsIncrementallyIdenticalTo(((XmlElementSyntax)updated.Nodes[0]).Content[0]));

There is a WithX method for every slot of every node, SyntaxFactory to build nodes from scratch, and RemoveNode with SyntaxRemoveOptions to take one out.

Nothing is carried over onto a replacement, including the trivia in front of the node being replaced. Ask for it with WithTriviaFrom when you want it.

Finding a node

With XPath, including namespaces:

var tree = XmlSyntaxTree.ParseText("<root xmlns:pkg='urn:test'><pkg:item pkg:version='1.0.0' /></root>");
var namespaces = new XmlNamespaceManager(new NameTable());
namespaces.AddNamespace("pkg", "urn:test");

var item = tree.GetRoot().SelectSingleSyntaxNode("//pkg:item", namespaces);
var version = tree.GetRoot().SelectSingleSyntaxNode("//pkg:item/@pkg:version", namespaces);

Assert.IsType<XmlEmptyElementSyntax>(item);
Assert.Equal("1.0.0", ((XmlAttributeSyntax)version!).Value);

SelectNodes returns XPathNavigators, SelectSyntaxNodes returns the syntax nodes behind them, and CreateNavigator() gives an XPathNavigator over the whole document. Or, without XPath: root.DescendantNodes(), DescendantTokens(), DescendantTrivia(), FindToken(position), FindNode(span).

Finding a node again after an edit

An annotation is a marker you attach to a node and find again in the tree an edit produced, wherever it ended up:

var root = XmlSyntaxTree.ParseText("<root><a>1</a><b>2</b></root>").GetRoot();
var a = root.DescendantNodes().OfType<XmlElementSyntax>().Single(element => element.Name == "a");

var marker = new SyntaxAnnotation();
var marked = root.ReplaceNode(a, a.WithAdditionalAnnotations(marker));

// Something else entirely: replace <b>, which rebuilds the path from <b> up to the root
var b = marked.DescendantNodes().OfType<XmlElementSyntax>().Single(element => element.Name == "b");
var edited = marked.ReplaceNode(b, b.WithInnerText("two"));

var found = edited.GetAnnotatedNodes(marker).Single();
Assert.Equal("<a>1</a>", found.ToFullString());

Diagnostics

Nothing is thrown for bad input; it is reported instead, in the order it appears in the text, with a location you can turn into line and character positions.

The checks are those of a non-validating XML processor: every well-formedness constraint of XML 1.0 (Fifth Edition) and of Namespaces in XML 1.0. A document System.Xml can read produces no diagnostic, with two deliberate exceptions where the specifications are stricter than it is — an encoding name that is not an EncName, and an element name with the xmlns prefix — and one where they are more permissive: any 1.x version is accepted.

var messages = new List<string>();
foreach (var diagnostic in XmlSyntaxTree.ParseText("<root>\n  <item>\n</root>").GetDiagnostics())
{
    var line = diagnostic.Location.GetLineSpan().Start;
    messages.Add($"{diagnostic.Id} ({line.Line},{line.Character}): {diagnostic.Message}");
}

Assert.Equal(["XML0001 (1,2): Missing end tag for 'item'."], messages);

An entity reference counts as declared when the internal subset declares it, or when the document has an external subset or a parameter entity reference, which could declare it where the parser cannot see.

Recovery

The parser keeps the shape the author most likely meant, so one mistake produces one diagnostic rather than a cascade:

• An end tag that matches an element further up closes the elements left open in between, each reported as missing its end tag. One that matches no open element is kept as XmlSkippedTextSyntax.
• A start tag missing its > still opens its element, with a missing GreaterThanToken.
• Text a tag cannot use — <a @ b="1"> — is kept as SkippedTextTrivia on the next token, and the element and the rest of its attributes are read as usual.
• An attribute value that meets what looks like the next tag before its closing quote stops there.
• A < that starts no markup, as in a < b, is skipped on its own rather than swallowing the text up to the next >.
• An unterminated XML declaration, document type declaration, or end tag stops at the next <.

Walking a tree

XmlSyntaxWalker visits a node and everything below it; XmlSyntaxVisitor dispatches on kind without recursing; XmlSyntaxRewriter builds a new tree from an old one.

private sealed class ElementCounter : XmlSyntaxWalker
{
    public int Count { get; private set; }

    public override void VisitElement(XmlElementSyntax node)
    {
        Count++;
        base.VisitElement(node);
    }
}

var counter = new ElementCounter();
counter.Visit(XmlSyntaxTree.ParseText("<root><a><b></b></a><c /></root>").GetRoot());

// <c /> is an XmlEmptyElementSyntax, so VisitElement does not count it
Assert.Equal(3, counter.Count);

Formatting

Formatter.Format lays a document out again, with XmlFormattingOptions controlling indentation, line breaks, attribute wrapping, the spacing inside tags, and whether elements written on one line stay that way. It is the one thing here that does reformat, and it is opt-in. It never changes text: an element that contains text keeps its content as written, and so does an element under xml:space="preserve".

Editing by text

var tree = XmlSyntaxTree.ParseText("<root><a /></root>");
var updated = tree.WithChanges(new TextChange(new TextSpan(6, 5), "<b />"));

Assert.Equal("<root><b /></root>", updated.GetRoot().ToFullString());

Coming from version 3

Version 4 moved the tree onto the same model Roslyn uses, which changed most of the API.

New in version 4: every character is a token, so Span and FullSpan differ where a token carries trivia; WithX on every slot; SyntaxAnnotation; RemoveNode/SyntaxRemoveOptions; XmlSyntaxWalker; FindToken/ FindNode; and structural sharing — an edit keeps every node it did not touch.

Notes

Replacing a node, token or trivium rebuilds only the spine from it to the root; WithChanges reparses, because an edit expressed as text can change how everything after it reads.

Nodes are shared between the trees an edit produces, so holding several versions of a document costs little more than holding one.

This is a parser, not a validator. It reads the shape of a document without applying a DTD or a schema.

The syntax keeps text exactly as written, references included; XmlAttributeSyntax.Value and XmlTextSyntax.Value are what an XML processor reads it as. Character references and the five predefined entities are resolved, line breaks are normalized to \n, and an attribute value turns each tab and line break into a space. Other entities are left as written, since resolving one means reading the DTD. WithValue escapes a tab or a line break as a character reference, so the value it sets is the value read back. Names do follow XML's own NameStartChar and NameChar productions, so a name may hold a combining mark, a middle dot, or a character from outside the basic plane, and a Unicode letter XML leaves out — ª, say — does not start one.