From d8b2db49438d8a10caf0dc7ffba6bf6894d91772 Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Tue, 9 Jun 2026 08:02:14 +0200 Subject: [PATCH] feat(parsing): EUC-KR (CP949) decoding with vendored index (#99) --- CHANGELOG.md | 2 +- README.md | 2 +- Sources/Parsing/ByteDecoder.swift | 4 + Sources/Parsing/EUCKRDecoder.swift | 43 ++++ Sources/Parsing/EUCKRTable1.swift | 336 +++++++++++++++++++++++++ Sources/Parsing/EUCKRTable2.swift | 336 +++++++++++++++++++++++++ Sources/Parsing/EUCKRTable3.swift | 336 +++++++++++++++++++++++++ Sources/Parsing/EUCKRTable4.swift | 336 +++++++++++++++++++++++++ Sources/Parsing/EUCKRTable5.swift | 336 +++++++++++++++++++++++++ Sources/Parsing/EUCKRTable6.swift | 336 +++++++++++++++++++++++++ Sources/Parsing/InputEncoding.swift | 1 + Sources/Parsing/StreamingDecoder.swift | 21 ++ Tests/EncodingTests.swift | 6 + 13 files changed, 2093 insertions(+), 2 deletions(-) create mode 100644 Sources/Parsing/EUCKRDecoder.swift create mode 100644 Sources/Parsing/EUCKRTable1.swift create mode 100644 Sources/Parsing/EUCKRTable2.swift create mode 100644 Sources/Parsing/EUCKRTable3.swift create mode 100644 Sources/Parsing/EUCKRTable4.swift create mode 100644 Sources/Parsing/EUCKRTable5.swift create mode 100644 Sources/Parsing/EUCKRTable6.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index f32abc7..fcd5fd5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,7 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added -- CJK multi-byte encodings (#99, in progress): the byte decoder gains a multi-byte path and the WHATWG **Shift-JIS** and **EUC-JP** algorithms, for both the whole-buffer and streaming decoders, selected by the XML declaration's `encoding` name. Shift-JIS covers ASCII, half-width katakana, and the JIS X 0208 plane; EUC-JP adds the `0x8E` half-width katakana and `0x8F` JIS X 0212 leads. The JIS X 0208 and 0212 indexes are vendored verbatim from the WHATWG `index-jis0208.txt`/`index-jis0212.txt`. The remaining CJK encodings (EUC-KR, GB2312/GBK) are tracked under the same issue. +- CJK multi-byte encodings (#99, in progress): the byte decoder gains a multi-byte path and the WHATWG **Shift-JIS** and **EUC-JP** algorithms, for both the whole-buffer and streaming decoders, selected by the XML declaration's `encoding` name. Shift-JIS covers ASCII, half-width katakana, and the JIS X 0208 plane; EUC-JP adds the `0x8E` half-width katakana and `0x8F` JIS X 0212 leads. EUC-KR (CP949/UHC) decodes Hangul and Hanja through the WHATWG `index-euc-kr`. The JIS X 0208/0212 and CP949 indexes are vendored verbatim from the corresponding WHATWG `index-*.txt` files. The remaining CJK encodings (GB2312/GBK) are tracked under the same issue. - HTML named character references: the complete WHATWG named character reference set (#84), all 2,125 semicolon-form references including the astral mathematical alphanumerics and the 93 two-codepoint references, vendored verbatim from the authoritative `entities.json` (replacing the previous 16-entry stopgap). Decoding stays longest-match with an optional trailing semicolon, on top of the existing numeric-reference validation. - HTML5 select content model (#83, stage 10): a `` closes the open one rather than nesting, and an ``/``/`