perf: make tree-to-tree transforms depth-safe (DTD defaults, canonicalization) (#352)
* perf(model): rebuild DTD-default application iteratively, not recursively applyingDefaults rebuilt the tree with a per-node recursive walk, so applying DTD attribute defaults to a deeply-nested document overflowed the stack (around 2000 deep on a Swift Task stack). It now rebuilds bottom-up through a new shared iterative spine, Node.rebuildingBottomUp, which transforms each node once its children are rebuilt via an explicit work stack. The same primitive will carry the other tree-to-tree transforms (canonical prefix rewrite, XSLT namespace fixup) in follow-ups. Defaults are applied identically: 1788 tests pass. Part of #350. * perf(canonical): serialize C14N iteratively, not recursively Canonicalizer.emit recursed on element children, so canonicalizing a deeply-nested document overflowed the stack (crashed around 5000 deep). The element serializer now drives a deferred-close work stack (the same shape as the main serializer): each step carries the in-scope and already-rendered namespace context, children push reversed to emit in document order, and a close step emits the end tag. The shallow document case and all namespace-rendering logic are unchanged. The canonical bytes are byte-identical: the C14N 1.0, 1.1, and exclusive conformance corpora pass, full suite 1789. A 50000-deep document now canonicalizes without crashing. The node-subset helpers moved to their own file to keep within the file-length limit. Part of #350.
This commit is contained in:
GitHub
parent
b672c49a84
commit
00c55e642f
@ -19,6 +19,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- Canonicalizing a document (`Canonicalizer`, the C14N form used for XML signatures) no longer recurses on the document's depth, so it no longer overflows the stack on a deeply-nested document (it crashed around 5000 deep). The serializer now drives a deferred-close work stack that threads each element's namespace context to its children. The canonical bytes are unchanged: the C14N 1.0/1.1 and exclusive conformance corpora pass. (#350)
|
||||
|
||||
- Applying DTD attribute defaults to a document (`parseApplyingInternalDTDDefaults`, and the defaults pass behind DTD validation) no longer recurses on the document's depth, so it no longer overflows the stack on a deeply-nested document. The tree is rebuilt bottom-up through a new shared iterative spine (`Node.rebuildingBottomUp`), which the other tree-to-tree transforms will adopt. (#350)
|
||||
|
||||
- The value-returning XPath result API (`PureXML.xpath`, `XPath.Query.evaluate`/`elements`/`strings`) is now linear for results that nest within one another, not quadratic. Each result was rebuilt as an independent value `Node` subtree, so selecting a deep chain of nested elements materialized the sum of their subtrees. The projection now shares a per-result-set memo keyed by tree-node identity, so a result reuses the already-built value `Node` of a deeper one by reference (copy-on-write); each tree node is projected once. `//a` over a 8000-deep chain drops from about 19 s to about 0.04 s, and the result set is unchanged. (#348)
|
||||
|
||||
- Sorting an XPath result into document order is now linear in the document, not quadratic in its depth. Each node's order key was its root path, an array as long as the node's depth, so ordering the descendants of a deeply-nested document was O(n log n x depth). The key is now a single O(1) index from a per-root pre-order numbering (built once per document and shared through the evaluation cache), with a band ordering an element before its namespace and attribute nodes and its children. A node-set spanning several documents is grouped by root. This linearizes the reference query path (`Query.nodes`, which XSLT and Schematron drive) on deep documents; the underlying subtree walk (`appendSubtree`) is now iterative too, so it no longer recurses. Document order is unchanged.
|
||||
|
||||
70
Sources/Canonical/CanonicalNodeSubset.swift
Normal file
70
Sources/Canonical/CanonicalNodeSubset.swift
Normal file
@ -0,0 +1,70 @@
|
||||
extension PureXML.Canonical.Canonicalizer {
|
||||
/// The apex element with its inherited `xml:*` attributes added, and (for
|
||||
/// inclusive mode) the namespaces in scope from omitted ancestors merged in
|
||||
/// as declarations so the apex renders its full namespace context.
|
||||
static func augmentedApex(
|
||||
_ apex: PureXML.Model.Element,
|
||||
inheritedXML: [PureXML.Model.Attribute],
|
||||
mergingNamespaces ancestorNamespaces: [String: String]?,
|
||||
stripApexBase: Bool,
|
||||
) -> PureXML.Model.Element {
|
||||
// Under 1.1, the apex's own xml:base is folded into the merged base in
|
||||
// inheritedXML, so drop the original to avoid emitting it twice.
|
||||
let own = stripApexBase
|
||||
? apex.attributes.filter { !($0.name.prefix == "xml" && $0.name.localName == "base") }
|
||||
: apex.attributes
|
||||
var attributes = own + inheritedXML
|
||||
if let ancestorNamespaces {
|
||||
let declared = Set(apex.attributes.map(\.name.description))
|
||||
for (prefix, uri) in ancestorNamespaces {
|
||||
let name = prefix.isEmpty ? "xmlns" : "xmlns:\(prefix)"
|
||||
if !declared.contains(name) { attributes.append(PureXML.Model.Attribute(name, uri)) }
|
||||
}
|
||||
}
|
||||
return PureXML.Model.Element(name: apex.name, attributes: attributes, children: apex.children)
|
||||
}
|
||||
|
||||
/// The namespace declarations in scope from a node's omitted ancestors,
|
||||
/// nearest ancestor winning (so an inner redeclaration is preserved).
|
||||
static func inScopeNamespaces(above node: PureXML.Model.TreeNode) -> [String: String] {
|
||||
var result: [String: String] = [:]
|
||||
var current = node.parent
|
||||
while let ancestor = current {
|
||||
for attribute in ancestor.attributes {
|
||||
guard let prefix = declaredPrefix(of: attribute.name), result[prefix] == nil else { continue }
|
||||
result[prefix] = attribute.value
|
||||
}
|
||||
current = ancestor.parent
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
/// The `xml:*` attributes in scope from a node's omitted ancestors that the
|
||||
/// apex does not already set, nearest ancestor winning.
|
||||
static func inheritedXMLAttributes(above node: PureXML.Model.TreeNode, apex: PureXML.Model.Element, mergeBase: Bool) -> [PureXML.Model.Attribute] {
|
||||
let present = Set(apex.attributes.map(\.name.description))
|
||||
if mergeBase { return inherited11XMLAttributes(above: node, apex: apex, present: present) }
|
||||
// Canonical XML 1.0: the nearest of every in-scope xml:* attribute the
|
||||
// apex does not already set.
|
||||
var seen: Set<String> = []
|
||||
var result: [PureXML.Model.Attribute] = []
|
||||
var current = node.parent
|
||||
while let ancestor = current {
|
||||
for attribute in ancestor.attributes where attribute.name.prefix == "xml" {
|
||||
let key = attribute.name.description
|
||||
guard !present.contains(key), seen.insert(key).inserted else { continue }
|
||||
result.append(attribute)
|
||||
}
|
||||
current = ancestor.parent
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
/// The prefix a namespace declaration binds (empty for the default
|
||||
/// namespace), or nil when the attribute is not a namespace declaration.
|
||||
static func declaredPrefix(of name: PureXML.Model.QualifiedName) -> String? {
|
||||
if name.prefix == nil, name.localName == "xmlns" { return "" }
|
||||
if name.prefix == "xmlns" { return name.localName }
|
||||
return nil
|
||||
}
|
||||
}
|
||||
6
Sources/Canonical/CanonicalStep.swift
Normal file
6
Sources/Canonical/CanonicalStep.swift
Normal file
@ -0,0 +1,6 @@
|
||||
/// One step of the iterative canonical serialization: a node to emit with its
|
||||
/// in-scope and already-rendered namespace context, or a deferred element close.
|
||||
enum CanonicalStep {
|
||||
case open(PureXML.Model.Node, inScope: [String: String], rendered: [String: String])
|
||||
case close(String)
|
||||
}
|
||||
@ -89,6 +89,35 @@ public extension PureXML.Canonical {
|
||||
rendered: [String: String],
|
||||
output: inout String,
|
||||
) {
|
||||
// Deferred-close work stack so a deeply-nested element does not
|
||||
// overflow the stack; children push reversed to emit in document order.
|
||||
var stack: [CanonicalStep] = [.open(.element(element), inScope: inScope, rendered: rendered)]
|
||||
while let step = stack.popLast() {
|
||||
switch step {
|
||||
case let .close(name):
|
||||
output += "</\(name)>"
|
||||
case let .open(node, inScope, rendered):
|
||||
guard case let .element(element) = node else {
|
||||
emitLeaf(node, into: &output)
|
||||
continue
|
||||
}
|
||||
let child = emitOpenTag(element, inScope: inScope, rendered: rendered, into: &output)
|
||||
stack.append(.close(element.name.description))
|
||||
for childNode in element.children.reversed() {
|
||||
stack.append(.open(childNode, inScope: child.inScope, rendered: child.rendered))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Emits an element's start tag (namespaces then attributes, in canonical
|
||||
/// order) and returns the namespace context its children inherit.
|
||||
private func emitOpenTag(
|
||||
_ element: PureXML.Model.Element,
|
||||
inScope: [String: String],
|
||||
rendered: [String: String],
|
||||
into output: inout String,
|
||||
) -> (inScope: [String: String], rendered: [String: String]) {
|
||||
let declarations = Self.namespaceDeclarations(element)
|
||||
var childInScope = inScope
|
||||
for (prefix, uri) in declarations {
|
||||
@ -109,10 +138,23 @@ public extension PureXML.Canonical {
|
||||
output += " \(attribute.name.description)=\"\(Self.escapeAttribute(attribute.value))\""
|
||||
}
|
||||
output += ">"
|
||||
for child in element.children {
|
||||
emit(child, inScope: childInScope, rendered: childRendered, output: &output)
|
||||
return (childInScope, childRendered)
|
||||
}
|
||||
|
||||
/// Emits a non-element node (text, CDATA, comment, or processing
|
||||
/// instruction) the same way the node-level `emit` does.
|
||||
private func emitLeaf(_ node: PureXML.Model.Node, into output: inout String) {
|
||||
switch node {
|
||||
case let .text(value), let .cdata(value):
|
||||
let text = options.trimTextNodes ? value.trimmingXMLWhitespace() : value
|
||||
output += Self.escapeText(text)
|
||||
case let .comment(value):
|
||||
if options.includeComments { output += "<!--\(value)-->" }
|
||||
case let .processingInstruction(target, data):
|
||||
output += data.isEmpty ? "<?\(target)?>" : "<?\(target) \(data)?>"
|
||||
case .element, .document:
|
||||
break
|
||||
}
|
||||
output += "</\(element.name.description)>"
|
||||
}
|
||||
|
||||
// MARK: Namespace selection
|
||||
@ -192,77 +234,6 @@ public extension PureXML.Canonical {
|
||||
return name.prefix != "xmlns" && !(name.prefix == nil && name.localName == "xmlns")
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: Node-subset context
|
||||
|
||||
/// The apex element with its inherited `xml:*` attributes added, and (for
|
||||
/// inclusive mode) the namespaces in scope from omitted ancestors merged in
|
||||
/// as declarations so the apex renders its full namespace context.
|
||||
private static func augmentedApex(
|
||||
_ apex: PureXML.Model.Element,
|
||||
inheritedXML: [PureXML.Model.Attribute],
|
||||
mergingNamespaces ancestorNamespaces: [String: String]?,
|
||||
stripApexBase: Bool,
|
||||
) -> PureXML.Model.Element {
|
||||
// Under 1.1, the apex's own xml:base is folded into the merged base in
|
||||
// inheritedXML, so drop the original to avoid emitting it twice.
|
||||
let own = stripApexBase
|
||||
? apex.attributes.filter { !($0.name.prefix == "xml" && $0.name.localName == "base") }
|
||||
: apex.attributes
|
||||
var attributes = own + inheritedXML
|
||||
if let ancestorNamespaces {
|
||||
let declared = Set(apex.attributes.map(\.name.description))
|
||||
for (prefix, uri) in ancestorNamespaces {
|
||||
let name = prefix.isEmpty ? "xmlns" : "xmlns:\(prefix)"
|
||||
if !declared.contains(name) { attributes.append(PureXML.Model.Attribute(name, uri)) }
|
||||
}
|
||||
}
|
||||
return PureXML.Model.Element(name: apex.name, attributes: attributes, children: apex.children)
|
||||
}
|
||||
|
||||
/// The namespace declarations in scope from a node's omitted ancestors,
|
||||
/// nearest ancestor winning (so an inner redeclaration is preserved).
|
||||
private static func inScopeNamespaces(above node: PureXML.Model.TreeNode) -> [String: String] {
|
||||
var result: [String: String] = [:]
|
||||
var current = node.parent
|
||||
while let ancestor = current {
|
||||
for attribute in ancestor.attributes {
|
||||
guard let prefix = declaredPrefix(of: attribute.name), result[prefix] == nil else { continue }
|
||||
result[prefix] = attribute.value
|
||||
}
|
||||
current = ancestor.parent
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
/// The `xml:*` attributes in scope from a node's omitted ancestors that the
|
||||
/// apex does not already set, nearest ancestor winning.
|
||||
private static func inheritedXMLAttributes(above node: PureXML.Model.TreeNode, apex: PureXML.Model.Element, mergeBase: Bool) -> [PureXML.Model.Attribute] {
|
||||
let present = Set(apex.attributes.map(\.name.description))
|
||||
if mergeBase { return inherited11XMLAttributes(above: node, apex: apex, present: present) }
|
||||
// Canonical XML 1.0: the nearest of every in-scope xml:* attribute the
|
||||
// apex does not already set.
|
||||
var seen: Set<String> = []
|
||||
var result: [PureXML.Model.Attribute] = []
|
||||
var current = node.parent
|
||||
while let ancestor = current {
|
||||
for attribute in ancestor.attributes where attribute.name.prefix == "xml" {
|
||||
let key = attribute.name.description
|
||||
guard !present.contains(key), seen.insert(key).inserted else { continue }
|
||||
result.append(attribute)
|
||||
}
|
||||
current = ancestor.parent
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
/// The prefix a namespace declaration binds (empty for the default
|
||||
/// namespace), or nil when the attribute is not a namespace declaration.
|
||||
private static func declaredPrefix(of name: PureXML.Model.QualifiedName) -> String? {
|
||||
if name.prefix == nil, name.localName == "xmlns" { return "" }
|
||||
if name.prefix == "xmlns" { return name.localName }
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
64
Sources/Model/NodeRebuild.swift
Normal file
64
Sources/Model/NodeRebuild.swift
Normal file
@ -0,0 +1,64 @@
|
||||
extension PureXML.Model.Node {
|
||||
/// Rebuilds this tree bottom-up without recursing on the tree's depth, so a
|
||||
/// transform over a deeply-nested document cannot overflow the stack. Each
|
||||
/// node is passed to `transform` once its children have been rebuilt, together
|
||||
/// with those rebuilt children (empty for a leaf); the transform returns the
|
||||
/// replacement node. An explicit work stack drives a post-order walk, so a
|
||||
/// parent is transformed only after all its descendants.
|
||||
///
|
||||
/// This is the shared spine for the tree-to-tree transforms (DTD default
|
||||
/// application, canonical prefix rewriting, and the like) that previously
|
||||
/// recursed one native frame per level.
|
||||
func rebuildingBottomUp(
|
||||
_ transform: (_ node: PureXML.Model.Node, _ rebuiltChildren: [PureXML.Model.Node]) -> PureXML.Model.Node,
|
||||
) -> PureXML.Model.Node {
|
||||
guard let rootChildren = Self.branchChildren(self) else {
|
||||
return transform(self, [])
|
||||
}
|
||||
let rootFrame = RebuildFrame(self, rootChildren)
|
||||
var stack: [RebuildFrame] = [rootFrame]
|
||||
while let frame = stack.last {
|
||||
guard frame.next < frame.children.count else {
|
||||
stack.removeLast()
|
||||
let rebuilt = transform(frame.node, frame.built)
|
||||
if let parent = stack.last {
|
||||
parent.built.append(rebuilt)
|
||||
} else {
|
||||
return rebuilt
|
||||
}
|
||||
continue
|
||||
}
|
||||
let child = frame.children[frame.next]
|
||||
frame.next += 1
|
||||
if let grandchildren = Self.branchChildren(child) {
|
||||
stack.append(RebuildFrame(child, grandchildren))
|
||||
} else {
|
||||
frame.built.append(transform(child, []))
|
||||
}
|
||||
}
|
||||
return transform(self, [])
|
||||
}
|
||||
|
||||
/// The children of a branch node (document or element), or nil for a leaf.
|
||||
private static func branchChildren(_ node: PureXML.Model.Node) -> [PureXML.Model.Node]? {
|
||||
switch node {
|
||||
case let .document(children): children
|
||||
case let .element(element): element.children
|
||||
default: nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A node being rebuilt bottom-up: its source children, the cursor into them, and
|
||||
/// the rebuilt children gathered so far.
|
||||
private final class RebuildFrame {
|
||||
let node: PureXML.Model.Node
|
||||
let children: [PureXML.Model.Node]
|
||||
var next = 0
|
||||
var built: [PureXML.Model.Node] = []
|
||||
|
||||
init(_ node: PureXML.Model.Node, _ children: [PureXML.Model.Node]) {
|
||||
self.node = node
|
||||
self.children = children
|
||||
}
|
||||
}
|
||||
@ -5,19 +5,28 @@ public extension PureXML.Validation.DTDSchema {
|
||||
/// values are the normalized defaults from the declaration, so an editor reads
|
||||
/// the same effective attributes a validating processor would see.
|
||||
func applyingDefaults(to node: PureXML.Model.Node) -> PureXML.Model.Node {
|
||||
switch node {
|
||||
case let .document(children):
|
||||
.document(children.map(applyingDefaults(to:)))
|
||||
case let .element(element):
|
||||
.element(applyingDefaults(to: element))
|
||||
case .text, .cdata, .comment, .processingInstruction:
|
||||
node
|
||||
// Rebuilt bottom-up (iterative) so a deeply-nested document does not
|
||||
// overflow the stack; each element gains its DTD-declared defaults once its
|
||||
// children are rebuilt.
|
||||
node.rebuildingBottomUp { original, rebuiltChildren in
|
||||
switch original {
|
||||
case let .element(element):
|
||||
var result = element
|
||||
result.children = rebuiltChildren
|
||||
return .element(withDefaults(result))
|
||||
case .document:
|
||||
return .document(rebuiltChildren)
|
||||
default:
|
||||
return original
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func applyingDefaults(to element: PureXML.Model.Element) -> PureXML.Model.Element {
|
||||
/// Adds the attributes `element`'s `<!ATTLIST>` declares with a default or
|
||||
/// `#FIXED` value and that the element omits. `result.children` is assumed
|
||||
/// already set by the caller.
|
||||
private func withDefaults(_ element: PureXML.Model.Element) -> PureXML.Model.Element {
|
||||
var result = element
|
||||
result.children = element.children.map { applyingDefaults(to: $0) }
|
||||
guard let declarations = attributes[element.name.description] else { return result }
|
||||
for declaration in declarations {
|
||||
guard let value = Self.defaultValue(declaration) else { continue }
|
||||
|
||||
38
Tests/TransformDepthSafetyTests.swift
Normal file
38
Tests/TransformDepthSafetyTests.swift
Normal file
@ -0,0 +1,38 @@
|
||||
import Testing
|
||||
@testable import PureXML
|
||||
|
||||
/// Tree-to-tree transforms must not recurse on a document's depth (#350). They
|
||||
/// rebuilt the tree with a per-node recursive walk, so transforming a deeply
|
||||
/// nested document overflowed the stack (as shallow as a couple of thousand deep
|
||||
/// on a Swift Task stack, which these tests run on). The walks are now iterative.
|
||||
@Suite("Transform depth safety")
|
||||
struct TransformDepthSafetyTests {
|
||||
private func deepXML(_ depth: Int, doctype: String = "") -> String {
|
||||
doctype + String(repeating: "<a>", count: depth) + "x" + String(repeating: "</a>", count: depth)
|
||||
}
|
||||
|
||||
@Test("applying DTD attribute defaults to a deep document does not overflow")
|
||||
func test_deepDTDDefaults() throws {
|
||||
let depth = 50000
|
||||
let xml = deepXML(depth, doctype: "<!DOCTYPE a [<!ATTLIST a kept CDATA \"on\">]>")
|
||||
let node = try PureXML.parseApplyingInternalDTDDefaults(xml, limits: .init(maxDepth: depth + 1, allowDoctype: true))
|
||||
// Every `a` element gained the defaulted `kept="on"` attribute; check the
|
||||
// outermost one without recursing.
|
||||
guard case let .document(roots) = node, let root = roots.first(where: { $0.element != nil })?.element else {
|
||||
Issue.record("expected a document with a root element")
|
||||
return
|
||||
}
|
||||
#expect(root.attributes.contains { $0.name.localName == "kept" && $0.value == "on" })
|
||||
}
|
||||
|
||||
@Test("canonicalizing a deep document does not overflow")
|
||||
func test_deepCanonicalize() throws {
|
||||
let depth = 50000
|
||||
let document = try PureXML.parse(deepXML(depth), limits: .init(maxDepth: depth + 1))
|
||||
let canonical = PureXML.Canonical.Canonicalizer(options: .inclusive).canonicalize(document)
|
||||
// Each level contributes "<a>"; the leaf text and the closes round it out.
|
||||
#expect(canonical.hasPrefix("<a>"))
|
||||
#expect(canonical.hasSuffix("</a>"))
|
||||
#expect(canonical.contains("x"))
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user