build(site): [SITE-07] reproduce historical Unicode data and ship source notices

This commit is contained in:
Harvey Zhao committed 2026-09-15 13:53:24 +08:00
1 parent 1fed96a909
commit 65368809ee
38 files changed
+78160 -9

No files matched your search

+2
View File
@@ -49,3 +49,5 @@ refactor/baselines/site-vendor/monaco-dom/** -text whitespace=-trailing-space,cr
docs/licenses/monaco-editor/core-dom/** -text whitespace=-trailing-space,cr-at-eol
docs/licenses/monaco-editor/core-path/** -text whitespace=-trailing-space,cr-at-eol
refactor/baselines/monaco-node-path-fixtures.json text eol=lf
refactor/baselines/site-vendor/monaco-unicode/** -text whitespace=-trailing-space,cr-at-eol
docs/licenses/monaco-editor/core-unicode/** -text whitespace=-trailing-space,cr-at-eol
+6
View File
@@ -125,6 +125,10 @@ Included component: WinJS-derived DOM helpers (Monaco core) unversioned adaptati
Source: https://github.com/winjs/winjs/tree/4329b1133b243d9ded3b5a1f98d096ee8e80e889
Included component: Unicode data (Monaco core) mixed historical inputs: 10.0.0d5, 13.0 drafts, Emoji 13.1
Source: https://github.com/alexdima/unicode-utils
- licenses/monaco-editor/LICENSE
- licenses/monaco-editor/ThirdPartyNotices.txt
- licenses/monaco-editor/codicons/LICENSE
@@ -154,6 +158,8 @@ Source: https://github.com/winjs/winjs/tree/4329b1133b243d9ded3b5a1f98d096ee8e80
- licenses/monaco-editor/core-path/ATTRIBUTION.md
- licenses/monaco-editor/core-dom/LICENSE.txt
- licenses/monaco-editor/core-dom/ATTRIBUTION.md
- licenses/monaco-editor/core-unicode/LICENSE.txt
- licenses/monaco-editor/core-unicode/ATTRIBUTION.md
## console legacy-vendor-with-TS-adapter
@@ -0,0 +1,15 @@
# Unicode data embedded in Monaco Editor 0.30.1
The editor and base worker contain generated Unicode data through Microsoft VS Code's vs/base/common/strings.ts. Existing Microsoft notices remain applicable to the surrounding code.
The original alexdima/unicode-utils generators reproduce all four selected data expressions from fixed VS Code commit 829382514cb1065f5ebb90f436e1c6103e153953 and both original Monaco source maps:
- RTL regular expression: UnicodeData-10.0.0d5.txt (draft), frozen at 5c256860b1e4e54b0375165b8451cf7cdabc953a. The version label comes from that commit's fetch recipe; this is not a claim about final Unicode 10.
- Emoji regular expression and imprecise code-point predicate: Emoji 13.1 emoji-test.txt and Unicode 13.0 UnicodeData.txt, frozen at 6d37cf4362203bfb5431c560cd8809b2cc3b51c1.
- Grapheme break tree: 5,034 integers from GraphemeBreakProperty-13.0.0d4.txt and the accompanying 2019 emoji-data.txt snapshot, frozen at 2068fd0695b2798d8debb3646c860f5d37ba75b2. Do not silently substitute final or later Unicode data.
Data files retain their original Unicode copyright headers. LICENSE.txt is the complete Unicode data/software license from unicode-org/unicodetools commit b1c21ef7e972ea0c783c19cf25494bde645fbf21, copyright 1991-2020 Unicode, Inc.; it is a fixed historical reference, not a claim that the generator distributed its own license file.
The generator repository's package.json identifies Alex Dima and declares MIT; that frozen tree has no standalone LICENSE file. Its lockfile resolves regexpu 3.3.0 despite conflicting dependency/devDependency ranges. Reproduction tools use the exact locked dependencies in an isolated cache; those tools and dependencies are not shipped with the site runtime.
The source/input hashes, recipes, archive integrity and reproduction command are maintained in refactor/baselines/monaco-unicode-provenance.json and scripts/site-vendor/monaco/README.md in the ArtPlayer source repository. This records data origins, not complete Unicode conformance or a complete historical Monaco build.
@@ -0,0 +1,46 @@
UNICODE, INC. LICENSE AGREEMENT - DATA FILES AND SOFTWARE
See Terms of Use for definitions of Unicode Inc.'s
Data Files and Software.
NOTICE TO USER: Carefully read the following legal agreement.
BY DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING UNICODE INC.'S
DATA FILES ("DATA FILES"), AND/OR SOFTWARE ("SOFTWARE"),
YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
TERMS AND CONDITIONS OF THIS AGREEMENT.
IF YOU DO NOT AGREE, DO NOT DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE
THE DATA FILES OR SOFTWARE.
COPYRIGHT AND PERMISSION NOTICE
Copyright © 1991-2020 Unicode, Inc. All rights reserved.
Distributed under the Terms of Use in https://www.unicode.org/copyright.html.
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Unicode data files and any associated documentation
(the "Data Files") or Unicode software and any associated documentation
(the "Software") to deal in the Data Files or Software
without restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, and/or sell copies of
the Data Files or Software, and to permit persons to whom the Data Files
or Software are furnished to do so, provided that either
(a) this copyright and permission notice appear with all copies
of the Data Files or Software, or
(b) this copyright and permission notice appear in associated
Documentation.
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT OF THIRD PARTY RIGHTS.
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS
NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL
DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THE DATA FILES OR SOFTWARE.
Except as contained in this notice, the name of a copyright holder
shall not be used in advertising or otherwise to promote the sale,
use or other dealings in these Data Files or Software without prior
written authorization of the copyright holder.
+3 -2
View File
@@ -45,7 +45,7 @@
"check:toolchain": "node scripts/check-toolchain.mjs",
"lint:fix": "eslint \"packages/*/{src,public,types,package.json}\" \"scripts/*.{js,mjs}\" \"packages/artplayer-vitepress/{browser,build}/**/*.ts\" \"scripts/{docs-smoke,editor-declarations,documentation,site-build,library,pages,site-vendor,browser-validation}/**/*.ts\" \"scripts/plugin/*.{js,ts}\" \"test/*\" \"docs/assets/ts/*\" \"types/*.d.ts\" \"playwright*.config.js\" --fix",
"check:plan": "node refactor/scripts/plan.mjs --check",
"test:node": "yarn test:unit && node --test test/toolchain.test.js test/build-docs.test.js test/package-check.test.js test/declarations.test.js test/editor-types.test.js test/coverage.test.js test/performance-report.test.js test/media-gate.test.js test/ci-summary.test.js test/package-runtime.test.js test/library-build.test.js test/site-loading.test.js test/documentation-pipeline.test.js test/site-build.test.js test/site-markdown.test.js test/site-editor.test.js test/plugin-scaffold.test.js test/dev-server.test.js test/pages-artifact.test.js test/site-notices.test.js test/monaco-provenance.test.js test/site-console.test.js test/vconsole-lifecycle.test.js test/browser-validation.test.js test/rollback-files.test.js test/pages-recovery.test.js",
"test:node": "yarn test:unit && node --test test/toolchain.test.js test/build-docs.test.js test/package-check.test.js test/declarations.test.js test/editor-types.test.js test/coverage.test.js test/performance-report.test.js test/media-gate.test.js test/ci-summary.test.js test/package-runtime.test.js test/library-build.test.js test/site-loading.test.js test/documentation-pipeline.test.js test/site-build.test.js test/site-markdown.test.js test/site-editor.test.js test/plugin-scaffold.test.js test/dev-server.test.js test/pages-artifact.test.js test/site-notices.test.js test/monaco-provenance.test.js test/monaco-unicode.test.js test/site-console.test.js test/vconsole-lifecycle.test.js test/browser-validation.test.js test/rollback-files.test.js test/pages-recovery.test.js",
"test:baseline": "node --test refactor/scripts/*.test.mjs",
"ci:check": "yarn check:toolchain --strict && yarn check:commits --report && yarn check:impact --report && yarn check:ci && yarn test:contracts && yarn check:contracts --report && yarn check:plan && yarn check:release-ledger && yarn check:rollback-inventory && yarn lint && yarn typecheck:library && yarn typecheck:scaffold && yarn typecheck:docs-tools && yarn check:docs-smoke && yarn check:editor-types && yarn check:llm && yarn check:vconsole && yarn check:console && yarn check:site-notices && yarn typecheck:site-assets && yarn check:site-assets && yarn check:types && yarn typecheck && yarn typecheck:react && yarn lint:react && yarn typecheck:vue && yarn lint:vue && yarn test",
"ci:build": "yarn build:types && yarn build all && yarn build:i18n && yarn build:ts && yarn build:llm && yarn build:vconsole && yarn build:console && yarn build:site-notices && yarn build:test && yarn build:docs && yarn test:imports",
@@ -174,7 +174,8 @@
"verify:monaco-contributions": "node scripts/site-vendor/monaco/reproduce-contributions.ts",
"verify:monaco-css": "node scripts/site-vendor/monaco/reproduce-css.ts",
"verify:monaco-node-path": "node scripts/site-vendor/monaco/reproduce-node-path.ts",
"verify:monaco-dom-origins": "node scripts/site-vendor/monaco/reproduce-dom-origins.ts"
"verify:monaco-dom-origins": "node scripts/site-vendor/monaco/reproduce-dom-origins.ts",
"verify:monaco-unicode": "node scripts/site-vendor/monaco/reproduce-unicode.ts"
},
"browserslist": "last 1 Chrome version",
"devDependencies": {
@@ -0,0 +1,396 @@
{
"schemaVersion": 1,
"task": "SITE-07",
"archives": [
{
"name": "monaco-editor",
"version": "0.30.1",
"tarball": "https://registry.npmjs.org/monaco-editor/-/monaco-editor-0.30.1.tgz",
"integrity": "sha512-B/y4+b2O5G2gjuxIFtCE2EkM17R2NM7/3F8x0qcPsqy4V83bitJTIO4TIeZpYlzu/xy6INiY/+84BEm6+7Cmzg==",
"sha256": "d7872ab742036036eca48cd54e43d751bede56742f0689bcf4049bcf701468c7"
},
{
"name": "monaco-editor-core",
"version": "0.30.1",
"tarball": "https://registry.npmjs.org/monaco-editor-core/-/monaco-editor-core-0.30.1.tgz",
"integrity": "sha512-WNxfchYafMZXVfysqg/ESW4MtOpYLPaIKrzudNlgwYvfYID+O/nwSZI5X+KxW84roVzHsNgFGyOOesfHFZwYUA==",
"sha256": "a3b004061373a7f6217eba6399f1a1de31970bee2eee693aada289035198c5ec"
}
],
"generatorArchives": [
{
"name": "ast-types",
"version": "0.9.6",
"tarball": "https://registry.npmjs.org/ast-types/-/ast-types-0.9.6.tgz",
"integrity": "sha512-qEdtR2UH78yyHX/AUNfXmJTlM48XoFZKBdwi1nzkI1mJL21cmbu0cvjxjpkXJ5NENMq42H+hNs8VLJcqXLerBQ==",
"sha256": "19c3f8d574f6005c7662238a5cea533ae54188784d25a6c28646e32f459b440b",
"lockedIntegrity": "sha1-ECyenpAF0+fjgpvwxPok7oYu6bk="
},
{
"name": "esprima",
"version": "3.1.3",
"tarball": "https://registry.npmjs.org/esprima/-/esprima-3.1.3.tgz",
"integrity": "sha512-AWwVMNxwhN8+NIPQzAQZCm7RkLC4RbM3B1OobMuyp3i+w73X57KCKaVIxaRZb+DYCojq7rspo+fmuQfAboyhFg==",
"sha256": "6ce53f45b7ccf15d62df7768a387a96ef1893014a2a7db68a3299029a6fd43cf",
"lockedIntegrity": "sha1-/cpRzuYTOJXjyI1TXOSdv/YqRjM="
},
{
"name": "jsesc",
"version": "0.5.0",
"tarball": "https://registry.npmjs.org/jsesc/-/jsesc-0.5.0.tgz",
"integrity": "sha512-uZz5UnB7u4T9LvwmFqXii7pZSouaRPorGs5who1Ip7VO0wxanFvBL7GkM6dTHlgX+jhBApRetaWpnDabOeTcnA==",
"sha256": "3e9a10467c80fe53f193fab92ae0ea5efd58decaa9a97f540fba18fd242f6674",
"lockedIntegrity": "sha1-597mbjXW/Bb3EP6R1c9p9w8IkR0="
},
{
"name": "private",
"version": "0.1.8",
"tarball": "https://registry.npmjs.org/private/-/private-0.1.8.tgz",
"integrity": "sha512-VvivMrbvd2nKkiG38qjULzlc+4Vx4wm/whI9pQD35YrARNnhxeiRktSOhSukRLFNlzg6Br/cJPet5J/u19r/mg==",
"sha256": "22ba10fa69b6ff5f28c3697594ac19b2540d9ae15b686869cfc533c3c9f23c1b",
"lockedIntegrity": "sha512-VvivMrbvd2nKkiG38qjULzlc+4Vx4wm/whI9pQD35YrARNnhxeiRktSOhSukRLFNlzg6Br/cJPet5J/u19r/mg=="
},
{
"name": "recast",
"version": "0.11.23",
"tarball": "https://registry.npmjs.org/recast/-/recast-0.11.23.tgz",
"integrity": "sha512-+nixG+3NugceyR8O1bLU45qs84JgI3+8EauyRZafLgC9XbdAOIVgwV1Pe2da0YzGo62KzWoZwUpVEQf6qNAXWA==",
"sha256": "134aa5059790cd0af1b76fcc02b758240a2cbd7608f9ba251de37fd35c6bae26",
"lockedIntegrity": "sha1-RR/TAEqx5N+bTktmN2sqIZEkYtM="
},
{
"name": "regenerate",
"version": "1.4.0",
"tarball": "https://registry.npmjs.org/regenerate/-/regenerate-1.4.0.tgz",
"integrity": "sha512-1G6jJVDWrt0rK99kBjvEtziZNCICAuvIPkSiUFIQxVP06RCVpq3dmDo2oi6ABpYaDYaTRr67BEhL8r1wgEZZKg==",
"sha256": "e90da6ab166ff03d0dcd2702e5f26c3696d6ef193a1820918ac34badc12af847",
"lockedIntegrity": "sha512-1G6jJVDWrt0rK99kBjvEtziZNCICAuvIPkSiUFIQxVP06RCVpq3dmDo2oi6ABpYaDYaTRr67BEhL8r1wgEZZKg=="
},
{
"name": "regenerate-unicode-properties",
"version": "2.0.0",
"tarball": "https://registry.npmjs.org/regenerate-unicode-properties/-/regenerate-unicode-properties-2.0.0.tgz",
"integrity": "sha512-nwsUrHTxYAgA6AyKvP3DBfRL6SrAL0/IRL/EXg/NYvwupss/97KYZK5B2govf5C/9GUGxKhgGtZ99v8fJTYFuQ==",
"sha256": "6b88e0426a53f613585573e26c6a9562a9bcd20ddff7200cb6ac5d7ab8beac6b",
"lockedIntegrity": "sha1-MFmsn7NIM69QpqAJgM9FwxmpF4Y="
},
{
"name": "regexpu",
"version": "3.3.0",
"tarball": "https://registry.npmjs.org/regexpu/-/regexpu-3.3.0.tgz",
"integrity": "sha512-Kcd/s6ss9yUzEhhrzsRzaZsp0Vn1CzaLyndBp7v8EsmNEiPCdith2nc3xYRuHvOiubhdLLa47O4e4ER//s4llA==",
"sha256": "1e35d657f807e560cbcc7e3f698c193a6e8fda45e7017884075b39b3d4aad04e",
"lockedIntegrity": "sha1-ib2Wh8E5o6BnpaF2+Jm7L4hnOAc="
},
{
"name": "regexpu-core",
"version": "3.3.0",
"tarball": "https://registry.npmjs.org/regexpu-core/-/regexpu-core-3.3.0.tgz",
"integrity": "sha512-yEaSrwX2BARhdZxw4bWUWbYQWCCCgngXKV/0eWS7cqh3dY0pSP6h91ZvnpLA/WscUNwFT5y57O/C5AkPJVWYKw==",
"sha256": "976066ff5bb73f7ba37ad97d2b3f7b9c531219b43ee7f007382013fcf5e8a44f",
"lockedIntegrity": "sha1-7fAAIU4osRe38JLIQAzNE9x091Q="
},
{
"name": "regjsgen",
"version": "0.3.0",
"tarball": "https://registry.npmjs.org/regjsgen/-/regjsgen-0.3.0.tgz",
"integrity": "sha512-HVnDMGSR2bwG7BQmTeyiarhsyGyfjO394lZU0T3j7nNEyqCW4SbeyW59ekcUghfruMa3vSOhFcBJ2V2OTRMotQ==",
"sha256": "d582217afc5615cda4f48db0809e33aecfe9ad8f6b59e0a2abce0495fdfcc1af",
"lockedIntegrity": "sha1-DuSj6SdkMM2iXx54nqbBW4ewy0M="
},
{
"name": "regjsparser",
"version": "0.2.1",
"tarball": "https://registry.npmjs.org/regjsparser/-/regjsparser-0.2.1.tgz",
"integrity": "sha512-tKQ9IpTLb426Y9bJ8+le9UG3miRf8vLlg2Ds9tF2wZ/5HaFv5lSRijMvy4PZYmETa260qiyyLVBFlfkf9/ATNw==",
"sha256": "ff4048ab6945c05307f0298503442f7d8bcc632080d7792e4c333d3fc5cc9c4a",
"lockedIntegrity": "sha1-w3h1U/rwTndcMCEC7zRtmVAA7Bw="
},
{
"name": "source-map",
"version": "0.5.7",
"tarball": "https://registry.npmjs.org/source-map/-/source-map-0.5.7.tgz",
"integrity": "sha512-LbrmJOMUSdEVxIKvdcJzQC+nQhe8FUZQTXQy6+I75skNgn3OoQ0DZA8YnFa7gp8tqtL3KPf1kmo0R5DoApeSGQ==",
"sha256": "e1de289bc493154f00a11cb4e363e0bb37619537d44b36b8112bba4924116b67",
"lockedIntegrity": "sha1-igOdLRAh0i0eoUyA2OpGi6LvP8w="
},
{
"name": "unicode-canonical-property-names",
"version": "1.0.3",
"tarball": "https://registry.npmjs.org/unicode-canonical-property-names/-/unicode-canonical-property-names-1.0.3.tgz",
"integrity": "sha512-DKdTLGSOgyvnLRHZdP7LJJFG/i0zta8ZIaqGExh+re+NQ2dxnjFaCiCC1+Ona5fPRkmb4O2uFAiTKzRNrJOa+w==",
"sha256": "390919055042b8c4214cae55fe589c69a79137c994d4ae7f1ad7ef20012e036a",
"lockedIntegrity": "sha1-Y7DwKXHEgSRT3yIevT75oWs8ouw="
},
{
"name": "unicode-match-property",
"version": "0.1.3",
"tarball": "https://registry.npmjs.org/unicode-match-property/-/unicode-match-property-0.1.3.tgz",
"integrity": "sha512-8sAYGMZ7FzXuRKAsR28EpmShqcQnR8B6VUXKGWSKxPypkx0zRIQP9N6FRDlcmWm8qDCcV9MJhKDgUBzrdz/vWQ==",
"sha256": "37596b06dfca35bdc7b9c58970f7d0c6c5ceff9702b7622b210bddeb0f91263a",
"lockedIntegrity": "sha1-NxxzCFIrZaLMTPu753UYf3eENSo="
},
{
"name": "unicode-match-property-value",
"version": "1.0.2",
"tarball": "https://registry.npmjs.org/unicode-match-property-value/-/unicode-match-property-value-1.0.2.tgz",
"integrity": "sha512-1gVylfXDNtKrI4C2bgHsQyx4XjgTfcb4YJ+DGkyFIdYW0RR842UM7GiHaZ5+4NxCuj6SntHfcqZoV11jzTDLNw==",
"sha256": "91847e893ca16d60ccfce052e65b4b0deeb9eb9a9a2ae24b927c1c6cb1e2a3cb",
"lockedIntegrity": "sha1-Vd4wCwQRKoy2Y8AQI0PAgnN3FMw="
},
{
"name": "unicode-property-aliases",
"version": "1.4.0",
"tarball": "https://registry.npmjs.org/unicode-property-aliases/-/unicode-property-aliases-1.4.0.tgz",
"integrity": "sha512-Wwwqwp89bM1GWmm/Lle7KSJKkUS0SRIK+KVHp0ryxy6k9Daa1fh/n8vt46wS5aswK7gzIWHWQBtRdXexgXdf3A==",
"sha256": "c26d9f8486b5622c15c826635fdd944ecd1ce94030c86f7a08740437a8b38def",
"lockedIntegrity": "sha512-Wwwqwp89bM1GWmm/Lle7KSJKkUS0SRIK+KVHp0ryxy6k9Daa1fh/n8vt46wS5aswK7gzIWHWQBtRdXexgXdf3A=="
},
{
"name": "unicode-property-value-aliases",
"version": "1.2.2",
"tarball": "https://registry.npmjs.org/unicode-property-value-aliases/-/unicode-property-value-aliases-1.2.2.tgz",
"integrity": "sha512-fByS+guPOhAbmjY8j1ZYOk4CBpzOAtaDcd19p5Xzg7O+4/gm3OXSdd2oEFZDfpgUalP1DJeVTSlcAkk/7L96aA==",
"sha256": "047566cde31e19801ea9e37bb363419ef69cc9f330facf749fca5e256de2ef89",
"lockedIntegrity": "sha1-EqN0b8ZvdfCKP0qNVngPifSay70="
}
],
"remotes": [
{
"source": "refactor/baselines/site-vendor/monaco-unicode/2017-UnicodeData.txt",
"sha256": "52423e4d7492167b62f518f68d54db88930abbbff7f11edfcaec8f726498cab1",
"gitBlobSha": "d89c64f526ae6e9ec74de10006101f43535bff31",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/d89c64f526ae6e9ec74de10006101f43535bff31",
"commit": "5c256860b1e4e54b0375165b8451cf7cdabc953a",
"upstream": "UnicodeData.txt"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/2019-GraphemeBreakProperty.txt",
"sha256": "2bd3c5e2d62701ff81fb3ec318e179a4618cabb1493b1e0dd2b4e7e56c5437c4",
"gitBlobSha": "6ee92f6eecaa58353d1b699b1226745b7d5d2552",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/6ee92f6eecaa58353d1b699b1226745b7d5d2552",
"commit": "2068fd0695b2798d8debb3646c860f5d37ba75b2",
"upstream": "GraphemeBreakProperty.txt"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/2019-emoji-data.txt",
"sha256": "e266a4e7e644da8923e7c3abbffed78597990570eda24eaa0f00f375cbdafd45",
"gitBlobSha": "6c0f22cd29ee8a5e036ebdb1d204e642c0d602ea",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/6c0f22cd29ee8a5e036ebdb1d204e642c0d602ea",
"commit": "2068fd0695b2798d8debb3646c860f5d37ba75b2",
"upstream": "emoji-data.txt"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/2021-emoji-test.txt",
"sha256": "8e3cc312a2a85115184738fa7ed97b3a9b3d0d221ffdaec3b22fd0314a18cd58",
"gitBlobSha": "d3c6d12bd5905ef3244844f6dd510637bfb3568c",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/d3c6d12bd5905ef3244844f6dd510637bfb3568c",
"commit": "6d37cf4362203bfb5431c560cd8809b2cc3b51c1",
"upstream": "emoji-test.txt"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/2017-fetch-data.js.txt",
"sha256": "06dbdae4bb518afed4ddca7592a7f5f42373e604b9687f36736ccc2fbf5a20f5",
"gitBlobSha": "99f3846ef581889df38c8775e8e2819eb2873846",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/99f3846ef581889df38c8775e8e2819eb2873846",
"commit": "5c256860b1e4e54b0375165b8451cf7cdabc953a",
"upstream": "fetch-data.js"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/2019-fetch-data.js.txt",
"sha256": "864b672817de67543bfc1f976374ac0e6bfbf9cf3cb86f040fa2b583afc7b115",
"gitBlobSha": "b83c1c0e24b0b107e0ea2834c681eabaa076ab05",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/b83c1c0e24b0b107e0ea2834c681eabaa076ab05",
"commit": "2068fd0695b2798d8debb3646c860f5d37ba75b2",
"upstream": "fetch-data.js"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/2021-fetch-data.js.txt",
"sha256": "395cccff5b1411746a876dad97e97eddd276823eee3c98694131e562e0d96a9e",
"gitBlobSha": "e97728526dbbcefc91f63ee9be34039f3a8e6286",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/e97728526dbbcefc91f63ee9be34039f3a8e6286",
"commit": "6d37cf4362203bfb5431c560cd8809b2cc3b51c1",
"upstream": "fetch-data.js"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/generate-rtl-test.js.txt",
"sha256": "c913beb0b0c8e23a1f3526dcb17df36e18bf4314da510991fc86ab42a2fd56cf",
"gitBlobSha": "e81988bafffdeed1a59ab75cddda878438384b75",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/e81988bafffdeed1a59ab75cddda878438384b75",
"commit": "6d37cf4362203bfb5431c560cd8809b2cc3b51c1",
"upstream": "generate-rtl-test.js"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/generate-emoji-test.js.txt",
"sha256": "621f35d443ad028f87437bb93c2042a1aff2fc56db30a36dd43389abd534cab6",
"gitBlobSha": "eafb836ef5028b9cbc7f1032381662ba17a82bf8",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/eafb836ef5028b9cbc7f1032381662ba17a82bf8",
"commit": "6d37cf4362203bfb5431c560cd8809b2cc3b51c1",
"upstream": "generate-emoji-test.js"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/generate-grapheme-break.js.txt",
"sha256": "e94ae28730f3d78576cc763c51d1e73b1a4c4710e6bd55e4c80150bafb7d10ed",
"gitBlobSha": "deed60dd5f7949a15bd362efda638474f2c72566",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/deed60dd5f7949a15bd362efda638474f2c72566",
"commit": "6d37cf4362203bfb5431c560cd8809b2cc3b51c1",
"upstream": "generate-grapheme-break.js"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/package.json.txt",
"sha256": "73553f3aa39e1830eab16253aea671efaf48a3da31be56b332a451fdb082c89f",
"gitBlobSha": "eb40f90fb8008930216f730ea63ea1783209a5b6",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/eb40f90fb8008930216f730ea63ea1783209a5b6",
"commit": "6d37cf4362203bfb5431c560cd8809b2cc3b51c1",
"upstream": "package.json"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/package-lock.json.txt",
"sha256": "858aa319fce68e7e2f55d42c5aade5c28f13fe0fbe199c1e6cd143fc311ddc7e",
"gitBlobSha": "4b31cf7ad0803292df18f831e34ba6dc867822bc",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/4b31cf7ad0803292df18f831e34ba6dc867822bc",
"commit": "6d37cf4362203bfb5431c560cd8809b2cc3b51c1",
"upstream": "package-lock.json"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/vscode-strings.txt",
"sha256": "7f947760385205ed17109ccec3ec31bafefd0292a9d872c933bea6042e8d4e77",
"gitBlobSha": "8a6bc0c28a252c3730a01cb388d8af41f01faf19",
"apiUrl": "https://api.github.com/repos/microsoft/vscode/git/blobs/8a6bc0c28a252c3730a01cb388d8af41f01faf19",
"commit": "829382514cb1065f5ebb90f436e1c6103e153953",
"upstream": "src/vs/base/common/strings.ts"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/LICENSE.txt",
"sha256": "80b381b2f157c31e7f3222bdb932bda9fbd7b335e888beec56820fb2993363de",
"gitBlobSha": "500dbd5463e43403fa163a8095828e7f6c1539c6",
"apiUrl": "https://api.github.com/repos/unicode-org/unicodetools/git/blobs/500dbd5463e43403fa163a8095828e7f6c1539c6",
"commit": "b1c21ef7e972ea0c783c19cf25494bde645fbf21",
"upstream": "LICENSE"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/2021-UnicodeData.txt",
"sha256": "bdbffbbfc8ad4d3a6d01b5891510458f3d36f7170422af4ea2bed3211a73e8bb",
"gitBlobSha": "e22f967bbab8f2477a43533a334e21ebc0728eda",
"apiUrl": "https://api.github.com/repos/alexdima/unicode-utils/git/blobs/e22f967bbab8f2477a43533a334e21ebc0728eda",
"commit": "6d37cf4362203bfb5431c560cd8809b2cc3b51c1",
"upstream": "UnicodeData.txt"
}
],
"vscodeSource": "refactor/baselines/site-vendor/monaco-unicode/vscode-strings.txt",
"packageSource": "refactor/baselines/site-vendor/monaco-unicode/package.json.txt",
"lockSource": "refactor/baselines/site-vendor/monaco-unicode/package-lock.json.txt",
"generators": [
{
"kind": "rtl",
"source": "refactor/baselines/site-vendor/monaco-unicode/generate-rtl-test.js.txt",
"inputs": {
"./UnicodeData.txt": "refactor/baselines/site-vendor/monaco-unicode/2017-UnicodeData.txt"
},
"outputs": [
"rtl-codes.txt",
"rtl-code-ranges.txt",
"rtl-code-ranges-fuzzy.txt"
]
},
{
"kind": "emoji",
"source": "refactor/baselines/site-vendor/monaco-unicode/generate-emoji-test.js.txt",
"inputs": {
"./emoji-test.txt": "refactor/baselines/site-vendor/monaco-unicode/2021-emoji-test.txt",
"./UnicodeData.txt": "refactor/baselines/site-vendor/monaco-unicode/2021-UnicodeData.txt"
},
"outputs": [
"emoji-codes.txt",
"emoji-code-ranges.txt",
"emoji-code-ranges-fuzzy.txt",
"emoji-missed.txt"
]
},
{
"kind": "grapheme",
"source": "refactor/baselines/site-vendor/monaco-unicode/generate-grapheme-break.js.txt",
"inputs": {
"./GraphemeBreakProperty.txt": "refactor/baselines/site-vendor/monaco-unicode/2019-GraphemeBreakProperty.txt",
"./emoji-data.txt": "refactor/baselines/site-vendor/monaco-unicode/2019-emoji-data.txt"
},
"outputs": []
}
],
"maps": [
{
"archive": "monaco-editor-core-0.30.1",
"member": "package/dev/vs/editor/editor.main.js.map",
"sha256": "22a1b5ff5f423adc4575004d2ae0ae231ba59516e6aa9c2fabe8a006c2730316",
"source": "file:///Users/alex/src/vscode/out-editor-src/vs/base/common/strings.ts"
},
{
"archive": "monaco-editor-core-0.30.1",
"member": "package/dev/vs/base/worker/workerMain.js.map",
"sha256": "9514f89d67e5baa44ddc6817edf75e675fb2371f5489bddcdb20386a72b4ef67",
"source": "file:///Users/alex/src/vscode/out-editor-src/vs/base/common/strings.ts"
}
],
"shipped": [
{
"archive": "monaco-editor-0.30.1",
"member": "package/min/vs/editor/editor.main.js",
"target": "docs/assets/js/vs/editor/editor.main.js",
"sha256": "05a50ed50bb013b6e7a82d25686186aa7bdff8b8209c7badab00e046b5e99618"
},
{
"archive": "monaco-editor-core-0.30.1",
"member": "package/min/vs/base/worker/workerMain.js",
"sha256": "3ebdb68f45f9fed1c587901c9db42dd70e8a552421e330f39116fc7b078ba67e",
"target": "docs/assets/js/vs/base/worker/workerMain.js"
}
],
"generated": {
"rtl": {
"sha256": "35fc83852ef1f0df43fab8378a534d5434858d79e27943ec932c36557c79260e",
"length": 488
},
"emoji": {
"sha256": "4f2e07efce92784638f0a8fde8d25abfef60e95517df8eb2b8ff9ba34cd6dd29",
"length": 173
},
"imprecise": {
"sha256": "233a3cb319d227bea46d424a312e9933fbef71ea0421ee0d9cde8b641609ab63",
"length": 350
},
"grapheme": {
"sha256": "3c464c0a0ccea5e151d0cb2786df211f72deb011f3fc6986b093d99242f0141c",
"length": 5034
}
},
"component": {
"name": "Unicode data (Monaco core)",
"version": "mixed historical inputs: 10.0.0d5, 13.0 drafts, Emoji 13.1",
"tarball": "https://github.com/alexdima/unicode-utils",
"assets": [
"docs/assets/js/vs/editor/editor.main.js",
"docs/assets/js/vs/base/worker/workerMain.js"
],
"notices": [
"docs/licenses/monaco-editor/core-unicode/LICENSE.txt",
"docs/licenses/monaco-editor/core-unicode/ATTRIBUTION.md"
]
},
"notices": [
{
"source": "refactor/baselines/site-vendor/monaco-unicode/LICENSE.txt",
"target": "docs/licenses/monaco-editor/core-unicode/LICENSE.txt",
"sha256": "80b381b2f157c31e7f3222bdb932bda9fbd7b335e888beec56820fb2993363de"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/ATTRIBUTION.md.txt",
"target": "docs/licenses/monaco-editor/core-unicode/ATTRIBUTION.md",
"sha256": "51d583153fd1ed701a2b8810b3b1994d337002f074255e75a7d06790a381a1a6"
}
],
"limits": [
"Data source versions are frozen recipe/snapshot identities, not a uniform Unicode release.",
"No full Monaco compilation or complete Unicode conformance claim.",
"Generator metadata declares MIT; its frozen tree has no standalone license file."
]
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,34 @@
var http = require('http');
var fs = require('fs');
function fetchFile(PATH, DEST) {
var options = {
hostname: 'www.unicode.org',
port: 80,
path: PATH,
method: 'GET'
};
var req = http.request(options, (res) => {
var dest = fs.createWriteStream(DEST);
console.log(`STATUS: ${res.statusCode}`);
console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
res.pipe(dest);
});
req.on('error', (e) => {
console.log(`problem with request: ${e.message}`);
});
// write data to request body
req.end();
}
// fetchFile(`/Public/9.0.0/ucd/UnicodeData.txt`, './UnicodeData.txt');
// 10.0.0 is still a WIP, but some emojis only show up in 10.0.0
fetchFile(`/Public/10.0.0/ucd/UnicodeData-10.0.0d5.txt`, './UnicodeData.txt');
fetchFile(`/Public/emoji/5.0/emoji-test.txt`, './emoji-test.txt');
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,767 @@
# emoji-data.txt
# Date: 2019-10-20, 14:06:35 GMT
# © 2019 Unicode®, Inc.
# Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the U.S. and other countries.
# For terms of use, see http://www.unicode.org/terms_of_use.html
#
# Emoji Data for UTS #51
# Version: 13.0
#
# For documentation and usage, see http://www.unicode.org/reports/tr51
#
# Format:
# <codepoint(s)> ; <property> # <comments>
# Note: there is no guarantee as to the structure of whitespace or comments
#
# Characters and sequences are listed in code point order. Users should be shown a more natural order.
# See the CLDR collation order for Emoji.
# ================================================
# All omitted code points have Emoji=No
# @missing: 0000..10FFFF ; Emoji ; No
0023 ; Emoji # E0.0 [1] (#️) number sign
002A ; Emoji # E0.0 [1] (*️) asterisk
0030..0039 ; Emoji # E0.0 [10] (0️..9️) digit zero..digit nine
00A9 ; Emoji # E2.0 [1] (©️) copyright
00AE ; Emoji # E2.0 [1] (®️) registered
203C ; Emoji # E2.0 [1] (‼️) double exclamation mark
2049 ; Emoji # E2.0 [1] (⁉️) exclamation question mark
2122 ; Emoji # E2.0 [1] (™️) trade mark
2139 ; Emoji # E2.0 [1] (ℹ️) information
2194..2199 ; Emoji # E2.0 [6] (↔️..↙️) left-right arrow..down-left arrow
21A9..21AA ; Emoji # E2.0 [2] (↩️..↪️) right arrow curving left..left arrow curving right
231A..231B ; Emoji # E2.0 [2] (⌚..⌛) watch..hourglass done
2328 ; Emoji # E2.0 [1] (⌨️) keyboard
23CF ; Emoji # E2.0 [1] (⏏️) eject button
23E9..23F3 ; Emoji # E2.0 [11] (⏩..⏳) fast-forward button..hourglass not done
23F8..23FA ; Emoji # E2.0 [3] (⏸️..⏺️) pause button..record button
24C2 ; Emoji # E2.0 [1] (Ⓜ️) circled M
25AA..25AB ; Emoji # E2.0 [2] (▪️..▫️) black small square..white small square
25B6 ; Emoji # E2.0 [1] (▶️) play button
25C0 ; Emoji # E2.0 [1] (◀️) reverse button
25FB..25FE ; Emoji # E2.0 [4] (◻️..◾) white medium square..black medium-small square
2600..2604 ; Emoji # E2.0 [5] (☀️..☄️) sun..comet
260E ; Emoji # E2.0 [1] (☎️) telephone
2611 ; Emoji # E2.0 [1] (☑️) check box with check
2614..2615 ; Emoji # E2.0 [2] (☔..☕) umbrella with rain drops..hot beverage
2618 ; Emoji # E2.0 [1] (☘️) shamrock
261D ; Emoji # E2.0 [1] (☝️) index pointing up
2620 ; Emoji # E2.0 [1] (☠️) skull and crossbones
2622..2623 ; Emoji # E2.0 [2] (☢️..☣️) radioactive..biohazard
2626 ; Emoji # E2.0 [1] (☦️) orthodox cross
262A ; Emoji # E2.0 [1] (☪️) star and crescent
262E..262F ; Emoji # E2.0 [2] (☮️..☯️) peace symbol..yin yang
2638..263A ; Emoji # E2.0 [3] (☸️..☺️) wheel of dharma..smiling face
2640 ; Emoji # E4.0 [1] (♀️) female sign
2642 ; Emoji # E4.0 [1] (♂️) male sign
2648..2653 ; Emoji # E2.0 [12] (♈..♓) Aries..Pisces
265F ; Emoji # E11.0 [1] (♟️) chess pawn
2660 ; Emoji # E2.0 [1] (♠️) spade suit
2663 ; Emoji # E2.0 [1] (♣️) club suit
2665..2666 ; Emoji # E2.0 [2] (♥️..♦️) heart suit..diamond suit
2668 ; Emoji # E2.0 [1] (♨️) hot springs
267B ; Emoji # E2.0 [1] (♻️) recycling symbol
267E ; Emoji # E11.0 [1] (♾️) infinity
267F ; Emoji # E2.0 [1] (♿) wheelchair symbol
2692..2694 ; Emoji # E2.0 [3] (⚒️..⚔️) hammer and pick..crossed swords
2695 ; Emoji # E4.0 [1] (⚕️) medical symbol
2696..2697 ; Emoji # E2.0 [2] (⚖️..⚗️) balance scale..alembic
2699 ; Emoji # E2.0 [1] (⚙️) gear
269B..269C ; Emoji # E2.0 [2] (⚛️..⚜️) atom symbol..fleur-de-lis
26A0..26A1 ; Emoji # E2.0 [2] (⚠️..⚡) warning..high voltage
26A7 ; Emoji # E13.0 [1] (⚧️) transgender symbol
26AA..26AB ; Emoji # E2.0 [2] (⚪..⚫) white circle..black circle
26B0..26B1 ; Emoji # E2.0 [2] (⚰️..⚱️) coffin..funeral urn
26BD..26BE ; Emoji # E2.0 [2] (⚽..⚾) soccer ball..baseball
26C4..26C5 ; Emoji # E2.0 [2] (⛄..⛅) snowman without snow..sun behind cloud
26C8 ; Emoji # E2.0 [1] (⛈️) cloud with lightning and rain
26CE..26CF ; Emoji # E2.0 [2] (⛎..⛏️) Ophiuchus..pick
26D1 ; Emoji # E2.0 [1] (⛑️) rescue worker’s helmet
26D3..26D4 ; Emoji # E2.0 [2] (⛓️..⛔) chains..no entry
26E9..26EA ; Emoji # E2.0 [2] (⛩️..⛪) shinto shrine..church
26F0..26F5 ; Emoji # E2.0 [6] (⛰️..⛵) mountain..sailboat
26F7..26FA ; Emoji # E2.0 [4] (⛷️..⛺) skier..tent
26FD ; Emoji # E2.0 [1] (⛽) fuel pump
2702 ; Emoji # E2.0 [1] (✂️) scissors
2705 ; Emoji # E2.0 [1] (✅) check mark button
2708..270D ; Emoji # E2.0 [6] (✈️..✍️) airplane..writing hand
270F ; Emoji # E2.0 [1] (✏️) pencil
2712 ; Emoji # E2.0 [1] (✒️) black nib
2714 ; Emoji # E2.0 [1] (✔️) check mark
2716 ; Emoji # E2.0 [1] (✖️) multiplication sign
271D ; Emoji # E2.0 [1] (✝️) latin cross
2721 ; Emoji # E2.0 [1] (✡️) star of David
2728 ; Emoji # E2.0 [1] (✨) sparkles
2733..2734 ; Emoji # E2.0 [2] (✳️..✴️) eight-spoked asterisk..eight-pointed star
2744 ; Emoji # E2.0 [1] (❄️) snowflake
2747 ; Emoji # E2.0 [1] (❇️) sparkle
274C ; Emoji # E2.0 [1] (❌) cross mark
274E ; Emoji # E2.0 [1] (❎) cross mark button
2753..2755 ; Emoji # E2.0 [3] (❓..❕) question mark..white exclamation mark
2757 ; Emoji # E2.0 [1] (❗) exclamation mark
2763..2764 ; Emoji # E2.0 [2] (❣️..❤️) heart exclamation..red heart
2795..2797 ; Emoji # E2.0 [3] (➕..➗) plus sign..division sign
27A1 ; Emoji # E2.0 [1] (➡️) right arrow
27B0 ; Emoji # E2.0 [1] (➰) curly loop
27BF ; Emoji # E2.0 [1] (➿) double curly loop
2934..2935 ; Emoji # E2.0 [2] (⤴️..⤵️) right arrow curving up..right arrow curving down
2B05..2B07 ; Emoji # E2.0 [3] (⬅️..⬇️) left arrow..down arrow
2B1B..2B1C ; Emoji # E2.0 [2] (⬛..⬜) black large square..white large square
2B50 ; Emoji # E2.0 [1] (⭐) star
2B55 ; Emoji # E2.0 [1] (⭕) hollow red circle
3030 ; Emoji # E2.0 [1] (〰️) wavy dash
303D ; Emoji # E2.0 [1] (〽️) part alternation mark
3297 ; Emoji # E2.0 [1] (㊗️) Japanese “congratulations” button
3299 ; Emoji # E2.0 [1] (㊙️) Japanese “secret” button
1F004 ; Emoji # E2.0 [1] (🀄) mahjong red dragon
1F0CF ; Emoji # E2.0 [1] (🃏) joker
1F170..1F171 ; Emoji # E2.0 [2] (🅰️..🅱️) A button (blood type)..B button (blood type)
1F17E..1F17F ; Emoji # E2.0 [2] (🅾️..🅿️) O button (blood type)..P button
1F18E ; Emoji # E2.0 [1] (🆎) AB button (blood type)
1F191..1F19A ; Emoji # E2.0 [10] (🆑..🆚) CL button..VS button
1F1E6..1F1FF ; Emoji # E0.0 [26] (🇦..🇿) regional indicator symbol letter a..regional indicator symbol letter z
1F201..1F202 ; Emoji # E2.0 [2] (🈁..🈂️) Japanese “here” button..Japanese “service charge” button
1F21A ; Emoji # E2.0 [1] (🈚) Japanese “free of charge” button
1F22F ; Emoji # E2.0 [1] (🈯) Japanese “reserved” button
1F232..1F23A ; Emoji # E2.0 [9] (🈲..🈺) Japanese “prohibited” button..Japanese “open for business” button
1F250..1F251 ; Emoji # E2.0 [2] (🉐..🉑) Japanese “bargain” button..Japanese “acceptable” button
1F300..1F321 ; Emoji # E2.0 [34] (🌀..🌡️) cyclone..thermometer
1F324..1F393 ; Emoji # E2.0 [112] (🌤️..🎓) sun behind small cloud..graduation cap
1F396..1F397 ; Emoji # E2.0 [2] (🎖️..🎗️) military medal..reminder ribbon
1F399..1F39B ; Emoji # E2.0 [3] (🎙️..🎛️) studio microphone..control knobs
1F39E..1F3F0 ; Emoji # E2.0 [83] (🎞️..🏰) film frames..castle
1F3F3..1F3F5 ; Emoji # E2.0 [3] (🏳️..🏵️) white flag..rosette
1F3F7..1F4FD ; Emoji # E2.0 [263] (🏷️..📽️) label..film projector
1F4FF..1F53D ; Emoji # E2.0 [63] (📿..🔽) prayer beads..downwards button
1F549..1F54E ; Emoji # E2.0 [6] (🕉️..🕎) om..menorah
1F550..1F567 ; Emoji # E2.0 [24] (🕐..🕧) one o’clock..twelve-thirty
1F56F..1F570 ; Emoji # E2.0 [2] (🕯️..🕰️) candle..mantelpiece clock
1F573..1F579 ; Emoji # E2.0 [7] (🕳️..🕹️) hole..joystick
1F57A ; Emoji # E4.0 [1] (🕺) man dancing
1F587 ; Emoji # E2.0 [1] (🖇️) linked paperclips
1F58A..1F58D ; Emoji # E2.0 [4] (🖊️..🖍️) pen..crayon
1F590 ; Emoji # E2.0 [1] (🖐️) hand with fingers splayed
1F595..1F596 ; Emoji # E2.0 [2] (🖕..🖖) middle finger..vulcan salute
1F5A4 ; Emoji # E4.0 [1] (🖤) black heart
1F5A5 ; Emoji # E2.0 [1] (🖥️) desktop computer
1F5A8 ; Emoji # E2.0 [1] (🖨️) printer
1F5B1..1F5B2 ; Emoji # E2.0 [2] (🖱️..🖲️) computer mouse..trackball
1F5BC ; Emoji # E2.0 [1] (🖼️) framed picture
1F5C2..1F5C4 ; Emoji # E2.0 [3] (🗂️..🗄️) card index dividers..file cabinet
1F5D1..1F5D3 ; Emoji # E2.0 [3] (🗑️..🗓️) wastebasket..spiral calendar
1F5DC..1F5DE ; Emoji # E2.0 [3] (🗜️..🗞️) clamp..rolled-up newspaper
1F5E1 ; Emoji # E2.0 [1] (🗡️) dagger
1F5E3 ; Emoji # E2.0 [1] (🗣️) speaking head
1F5E8 ; Emoji # E2.0 [1] (🗨️) left speech bubble
1F5EF ; Emoji # E2.0 [1] (🗯️) right anger bubble
1F5F3 ; Emoji # E2.0 [1] (🗳️) ballot box with ballot
1F5FA..1F64F ; Emoji # E2.0 [86] (🗺️..🙏) world map..folded hands
1F680..1F6C5 ; Emoji # E2.0 [70] (🚀..🛅) rocket..left luggage
1F6CB..1F6D0 ; Emoji # E2.0 [6] (🛋️..🛐) couch and lamp..place of worship
1F6D1..1F6D2 ; Emoji # E4.0 [2] (🛑..🛒) stop sign..shopping cart
1F6D5 ; Emoji # E12.1 [1] (🛕) hindu temple
1F6D6..1F6D7 ; Emoji # E13.0 [2] (🛖..🛗) hut..elevator
1F6E0..1F6E5 ; Emoji # E2.0 [6] (🛠️..🛥️) hammer and wrench..motor boat
1F6E9 ; Emoji # E2.0 [1] (🛩️) small airplane
1F6EB..1F6EC ; Emoji # E2.0 [2] (🛫..🛬) airplane departure..airplane arrival
1F6F0 ; Emoji # E2.0 [1] (🛰️) satellite
1F6F3 ; Emoji # E2.0 [1] (🛳️) passenger ship
1F6F4..1F6F6 ; Emoji # E4.0 [3] (🛴..🛶) kick scooter..canoe
1F6F7..1F6F8 ; Emoji # E5.0 [2] (🛷..🛸) sled..flying saucer
1F6F9 ; Emoji # E11.0 [1] (🛹) skateboard
1F6FA ; Emoji # E12.1 [1] (🛺) auto rickshaw
1F6FB..1F6FC ; Emoji # E13.0 [2] (🛻..🛼) pickup truck..roller skate
1F7E0..1F7EB ; Emoji # E12.1 [12] (🟠..🟫) orange circle..brown square
1F90C ; Emoji # E13.0 [1] (🤌) pinched fingers
1F90D..1F90F ; Emoji # E12.1 [3] (🤍..🤏) white heart..pinching hand
1F910..1F918 ; Emoji # E2.0 [9] (🤐..🤘) zipper-mouth face..sign of the horns
1F919..1F91E ; Emoji # E4.0 [6] (🤙..🤞) call me hand..crossed fingers
1F91F ; Emoji # E5.0 [1] (🤟) love-you gesture
1F920..1F927 ; Emoji # E4.0 [8] (🤠..🤧) cowboy hat face..sneezing face
1F928..1F92F ; Emoji # E5.0 [8] (🤨..🤯) face with raised eyebrow..exploding head
1F930 ; Emoji # E4.0 [1] (🤰) pregnant woman
1F931..1F932 ; Emoji # E5.0 [2] (🤱..🤲) breast-feeding..palms up together
1F933..1F93A ; Emoji # E4.0 [8] (🤳..🤺) selfie..person fencing
1F93C..1F93E ; Emoji # E4.0 [3] (🤼..🤾) people wrestling..person playing handball
1F93F ; Emoji # E12.1 [1] (🤿) diving mask
1F940..1F945 ; Emoji # E4.0 [6] (🥀..🥅) wilted flower..goal net
1F947..1F94B ; Emoji # E4.0 [5] (🥇..🥋) 1st place medal..martial arts uniform
1F94C ; Emoji # E5.0 [1] (🥌) curling stone
1F94D..1F94F ; Emoji # E11.0 [3] (🥍..🥏) lacrosse..flying disc
1F950..1F95E ; Emoji # E4.0 [15] (🥐..🥞) croissant..pancakes
1F95F..1F96B ; Emoji # E5.0 [13] (🥟..🥫) dumpling..canned food
1F96C..1F970 ; Emoji # E11.0 [5] (🥬..🥰) leafy green..smiling face with hearts
1F971 ; Emoji # E12.1 [1] (🥱) yawning face
1F972 ; Emoji # E13.0 [1] (🥲) smiling face with tear
1F973..1F976 ; Emoji # E11.0 [4] (🥳..🥶) partying face..cold face
1F977..1F978 ; Emoji # E13.0 [2] (🥷..🥸) ninja..disguised face
1F97A ; Emoji # E11.0 [1] (🥺) pleading face
1F97B ; Emoji # E12.1 [1] (🥻) sari
1F97C..1F97F ; Emoji # E11.0 [4] (🥼..🥿) lab coat..flat shoe
1F980..1F984 ; Emoji # E2.0 [5] (🦀..🦄) crab..unicorn
1F985..1F991 ; Emoji # E4.0 [13] (🦅..🦑) eagle..squid
1F992..1F997 ; Emoji # E5.0 [6] (🦒..🦗) giraffe..cricket
1F998..1F9A2 ; Emoji # E11.0 [11] (🦘..🦢) kangaroo..swan
1F9A3..1F9A4 ; Emoji # E13.0 [2] (🦣..🦤) mammoth..dodo
1F9A5..1F9AA ; Emoji # E12.1 [6] (🦥..🦪) sloth..oyster
1F9AB..1F9AD ; Emoji # E13.0 [3] (🦫..🦭) beaver..seal
1F9AE..1F9AF ; Emoji # E12.1 [2] (🦮..🦯) guide dog..probing cane
1F9B0..1F9B9 ; Emoji # E11.0 [10] (🦰..🦹) red hair..supervillain
1F9BA..1F9BF ; Emoji # E12.1 [6] (🦺..🦿) safety vest..mechanical leg
1F9C0 ; Emoji # E2.0 [1] (🧀) cheese wedge
1F9C1..1F9C2 ; Emoji # E11.0 [2] (🧁..🧂) cupcake..salt
1F9C3..1F9CA ; Emoji # E12.1 [8] (🧃..🧊) beverage box..ice
1F9CB ; Emoji # E13.0 [1] (🧋) bubble tea
1F9CD..1F9CF ; Emoji # E12.1 [3] (🧍..🧏) person standing..deaf person
1F9D0..1F9E6 ; Emoji # E5.0 [23] (🧐..🧦) face with monocle..socks
1F9E7..1F9FF ; Emoji # E11.0 [25] (🧧..🧿) red envelope..nazar amulet
1FA70..1FA73 ; Emoji # E12.1 [4] (🩰..🩳) ballet shoes..shorts
1FA74 ; Emoji # E13.0 [1] (🩴) thong sandal
1FA78..1FA7A ; Emoji # E12.1 [3] (🩸..🩺) drop of blood..stethoscope
1FA80..1FA82 ; Emoji # E12.1 [3] (🪀..🪂) yo-yo..parachute
1FA83..1FA86 ; Emoji # E13.0 [4] (🪃..🪆) boomerang..nesting dolls
1FA90..1FA95 ; Emoji # E12.1 [6] (🪐..🪕) ringed planet..banjo
1FA96..1FAA8 ; Emoji # E13.0 [19] (🪖..🪨) military helmet..rock
1FAB0..1FAB6 ; Emoji # E13.0 [7] (🪰..🪶) fly..feather
1FAC0..1FAC2 ; Emoji # E13.0 [3] (🫀..🫂) anatomical heart..people hugging
1FAD0..1FAD6 ; Emoji # E13.0 [7] (🫐..🫖) blueberries..teapot
# Total elements: 1367
# ================================================
# All omitted code points have Emoji_Presentation=No
# @missing: 0000..10FFFF ; Emoji_Presentation ; No
231A..231B ; Emoji_Presentation # E2.0 [2] (⌚..⌛) watch..hourglass done
23E9..23EC ; Emoji_Presentation # E2.0 [4] (⏩..⏬) fast-forward button..fast down button
23F0 ; Emoji_Presentation # E2.0 [1] (⏰) alarm clock
23F3 ; Emoji_Presentation # E2.0 [1] (⏳) hourglass not done
25FD..25FE ; Emoji_Presentation # E2.0 [2] (◽..◾) white medium-small square..black medium-small square
2614..2615 ; Emoji_Presentation # E2.0 [2] (☔..☕) umbrella with rain drops..hot beverage
2648..2653 ; Emoji_Presentation # E2.0 [12] (♈..♓) Aries..Pisces
267F ; Emoji_Presentation # E2.0 [1] (♿) wheelchair symbol
2693 ; Emoji_Presentation # E2.0 [1] (⚓) anchor
26A1 ; Emoji_Presentation # E2.0 [1] (⚡) high voltage
26AA..26AB ; Emoji_Presentation # E2.0 [2] (⚪..⚫) white circle..black circle
26BD..26BE ; Emoji_Presentation # E2.0 [2] (⚽..⚾) soccer ball..baseball
26C4..26C5 ; Emoji_Presentation # E2.0 [2] (⛄..⛅) snowman without snow..sun behind cloud
26CE ; Emoji_Presentation # E2.0 [1] (⛎) Ophiuchus
26D4 ; Emoji_Presentation # E2.0 [1] (⛔) no entry
26EA ; Emoji_Presentation # E2.0 [1] (⛪) church
26F2..26F3 ; Emoji_Presentation # E2.0 [2] (⛲..⛳) fountain..flag in hole
26F5 ; Emoji_Presentation # E2.0 [1] (⛵) sailboat
26FA ; Emoji_Presentation # E2.0 [1] (⛺) tent
26FD ; Emoji_Presentation # E2.0 [1] (⛽) fuel pump
2705 ; Emoji_Presentation # E2.0 [1] (✅) check mark button
270A..270B ; Emoji_Presentation # E2.0 [2] (✊..✋) raised fist..raised hand
2728 ; Emoji_Presentation # E2.0 [1] (✨) sparkles
274C ; Emoji_Presentation # E2.0 [1] (❌) cross mark
274E ; Emoji_Presentation # E2.0 [1] (❎) cross mark button
2753..2755 ; Emoji_Presentation # E2.0 [3] (❓..❕) question mark..white exclamation mark
2757 ; Emoji_Presentation # E2.0 [1] (❗) exclamation mark
2795..2797 ; Emoji_Presentation # E2.0 [3] (➕..➗) plus sign..division sign
27B0 ; Emoji_Presentation # E2.0 [1] (➰) curly loop
27BF ; Emoji_Presentation # E2.0 [1] (➿) double curly loop
2B1B..2B1C ; Emoji_Presentation # E2.0 [2] (⬛..⬜) black large square..white large square
2B50 ; Emoji_Presentation # E2.0 [1] (⭐) star
2B55 ; Emoji_Presentation # E2.0 [1] (⭕) hollow red circle
1F004 ; Emoji_Presentation # E2.0 [1] (🀄) mahjong red dragon
1F0CF ; Emoji_Presentation # E2.0 [1] (🃏) joker
1F18E ; Emoji_Presentation # E2.0 [1] (🆎) AB button (blood type)
1F191..1F19A ; Emoji_Presentation # E2.0 [10] (🆑..🆚) CL button..VS button
1F1E6..1F1FF ; Emoji_Presentation # E0.0 [26] (🇦..🇿) regional indicator symbol letter a..regional indicator symbol letter z
1F201 ; Emoji_Presentation # E2.0 [1] (🈁) Japanese “here” button
1F21A ; Emoji_Presentation # E2.0 [1] (🈚) Japanese “free of charge” button
1F22F ; Emoji_Presentation # E2.0 [1] (🈯) Japanese “reserved” button
1F232..1F236 ; Emoji_Presentation # E2.0 [5] (🈲..🈶) Japanese “prohibited” button..Japanese “not free of charge” button
1F238..1F23A ; Emoji_Presentation # E2.0 [3] (🈸..🈺) Japanese “application” button..Japanese “open for business” button
1F250..1F251 ; Emoji_Presentation # E2.0 [2] (🉐..🉑) Japanese “bargain” button..Japanese “acceptable” button
1F300..1F320 ; Emoji_Presentation # E2.0 [33] (🌀..🌠) cyclone..shooting star
1F32D..1F335 ; Emoji_Presentation # E2.0 [9] (🌭..🌵) hot dog..cactus
1F337..1F37C ; Emoji_Presentation # E2.0 [70] (🌷..🍼) tulip..baby bottle
1F37E..1F393 ; Emoji_Presentation # E2.0 [22] (🍾..🎓) bottle with popping cork..graduation cap
1F3A0..1F3CA ; Emoji_Presentation # E2.0 [43] (🎠..🏊) carousel horse..person swimming
1F3CF..1F3D3 ; Emoji_Presentation # E2.0 [5] (🏏..🏓) cricket game..ping pong
1F3E0..1F3F0 ; Emoji_Presentation # E2.0 [17] (🏠..🏰) house..castle
1F3F4 ; Emoji_Presentation # E2.0 [1] (🏴) black flag
1F3F8..1F43E ; Emoji_Presentation # E2.0 [71] (🏸..🐾) badminton..paw prints
1F440 ; Emoji_Presentation # E2.0 [1] (👀) eyes
1F442..1F4FC ; Emoji_Presentation # E2.0 [187] (👂..📼) ear..videocassette
1F4FF..1F53D ; Emoji_Presentation # E2.0 [63] (📿..🔽) prayer beads..downwards button
1F54B..1F54E ; Emoji_Presentation # E2.0 [4] (🕋..🕎) kaaba..menorah
1F550..1F567 ; Emoji_Presentation # E2.0 [24] (🕐..🕧) one o’clock..twelve-thirty
1F57A ; Emoji_Presentation # E4.0 [1] (🕺) man dancing
1F595..1F596 ; Emoji_Presentation # E2.0 [2] (🖕..🖖) middle finger..vulcan salute
1F5A4 ; Emoji_Presentation # E4.0 [1] (🖤) black heart
1F5FB..1F64F ; Emoji_Presentation # E2.0 [85] (🗻..🙏) mount fuji..folded hands
1F680..1F6C5 ; Emoji_Presentation # E2.0 [70] (🚀..🛅) rocket..left luggage
1F6CC ; Emoji_Presentation # E2.0 [1] (🛌) person in bed
1F6D0 ; Emoji_Presentation # E2.0 [1] (🛐) place of worship
1F6D1..1F6D2 ; Emoji_Presentation # E4.0 [2] (🛑..🛒) stop sign..shopping cart
1F6D5 ; Emoji_Presentation # E12.1 [1] (🛕) hindu temple
1F6D6..1F6D7 ; Emoji_Presentation # E13.0 [2] (🛖..🛗) hut..elevator
1F6EB..1F6EC ; Emoji_Presentation # E2.0 [2] (🛫..🛬) airplane departure..airplane arrival
1F6F4..1F6F6 ; Emoji_Presentation # E4.0 [3] (🛴..🛶) kick scooter..canoe
1F6F7..1F6F8 ; Emoji_Presentation # E5.0 [2] (🛷..🛸) sled..flying saucer
1F6F9 ; Emoji_Presentation # E11.0 [1] (🛹) skateboard
1F6FA ; Emoji_Presentation # E12.1 [1] (🛺) auto rickshaw
1F6FB..1F6FC ; Emoji_Presentation # E13.0 [2] (🛻..🛼) pickup truck..roller skate
1F7E0..1F7EB ; Emoji_Presentation # E12.1 [12] (🟠..🟫) orange circle..brown square
1F90C ; Emoji_Presentation # E13.0 [1] (🤌) pinched fingers
1F90D..1F90F ; Emoji_Presentation # E12.1 [3] (🤍..🤏) white heart..pinching hand
1F910..1F918 ; Emoji_Presentation # E2.0 [9] (🤐..🤘) zipper-mouth face..sign of the horns
1F919..1F91E ; Emoji_Presentation # E4.0 [6] (🤙..🤞) call me hand..crossed fingers
1F91F ; Emoji_Presentation # E5.0 [1] (🤟) love-you gesture
1F920..1F927 ; Emoji_Presentation # E4.0 [8] (🤠..🤧) cowboy hat face..sneezing face
1F928..1F92F ; Emoji_Presentation # E5.0 [8] (🤨..🤯) face with raised eyebrow..exploding head
1F930 ; Emoji_Presentation # E4.0 [1] (🤰) pregnant woman
1F931..1F932 ; Emoji_Presentation # E5.0 [2] (🤱..🤲) breast-feeding..palms up together
1F933..1F93A ; Emoji_Presentation # E4.0 [8] (🤳..🤺) selfie..person fencing
1F93C..1F93E ; Emoji_Presentation # E4.0 [3] (🤼..🤾) people wrestling..person playing handball
1F93F ; Emoji_Presentation # E12.1 [1] (🤿) diving mask
1F940..1F945 ; Emoji_Presentation # E4.0 [6] (🥀..🥅) wilted flower..goal net
1F947..1F94B ; Emoji_Presentation # E4.0 [5] (🥇..🥋) 1st place medal..martial arts uniform
1F94C ; Emoji_Presentation # E5.0 [1] (🥌) curling stone
1F94D..1F94F ; Emoji_Presentation # E11.0 [3] (🥍..🥏) lacrosse..flying disc
1F950..1F95E ; Emoji_Presentation # E4.0 [15] (🥐..🥞) croissant..pancakes
1F95F..1F96B ; Emoji_Presentation # E5.0 [13] (🥟..🥫) dumpling..canned food
1F96C..1F970 ; Emoji_Presentation # E11.0 [5] (🥬..🥰) leafy green..smiling face with hearts
1F971 ; Emoji_Presentation # E12.1 [1] (🥱) yawning face
1F972 ; Emoji_Presentation # E13.0 [1] (🥲) smiling face with tear
1F973..1F976 ; Emoji_Presentation # E11.0 [4] (🥳..🥶) partying face..cold face
1F977..1F978 ; Emoji_Presentation # E13.0 [2] (🥷..🥸) ninja..disguised face
1F97A ; Emoji_Presentation # E11.0 [1] (🥺) pleading face
1F97B ; Emoji_Presentation # E12.1 [1] (🥻) sari
1F97C..1F97F ; Emoji_Presentation # E11.0 [4] (🥼..🥿) lab coat..flat shoe
1F980..1F984 ; Emoji_Presentation # E2.0 [5] (🦀..🦄) crab..unicorn
1F985..1F991 ; Emoji_Presentation # E4.0 [13] (🦅..🦑) eagle..squid
1F992..1F997 ; Emoji_Presentation # E5.0 [6] (🦒..🦗) giraffe..cricket
1F998..1F9A2 ; Emoji_Presentation # E11.0 [11] (🦘..🦢) kangaroo..swan
1F9A3..1F9A4 ; Emoji_Presentation # E13.0 [2] (🦣..🦤) mammoth..dodo
1F9A5..1F9AA ; Emoji_Presentation # E12.1 [6] (🦥..🦪) sloth..oyster
1F9AB..1F9AD ; Emoji_Presentation # E13.0 [3] (🦫..🦭) beaver..seal
1F9AE..1F9AF ; Emoji_Presentation # E12.1 [2] (🦮..🦯) guide dog..probing cane
1F9B0..1F9B9 ; Emoji_Presentation # E11.0 [10] (🦰..🦹) red hair..supervillain
1F9BA..1F9BF ; Emoji_Presentation # E12.1 [6] (🦺..🦿) safety vest..mechanical leg
1F9C0 ; Emoji_Presentation # E2.0 [1] (🧀) cheese wedge
1F9C1..1F9C2 ; Emoji_Presentation # E11.0 [2] (🧁..🧂) cupcake..salt
1F9C3..1F9CA ; Emoji_Presentation # E12.1 [8] (🧃..🧊) beverage box..ice
1F9CB ; Emoji_Presentation # E13.0 [1] (🧋) bubble tea
1F9CD..1F9CF ; Emoji_Presentation # E12.1 [3] (🧍..🧏) person standing..deaf person
1F9D0..1F9E6 ; Emoji_Presentation # E5.0 [23] (🧐..🧦) face with monocle..socks
1F9E7..1F9FF ; Emoji_Presentation # E11.0 [25] (🧧..🧿) red envelope..nazar amulet
1FA70..1FA73 ; Emoji_Presentation # E12.1 [4] (🩰..🩳) ballet shoes..shorts
1FA74 ; Emoji_Presentation # E13.0 [1] (🩴) thong sandal
1FA78..1FA7A ; Emoji_Presentation # E12.1 [3] (🩸..🩺) drop of blood..stethoscope
1FA80..1FA82 ; Emoji_Presentation # E12.1 [3] (🪀..🪂) yo-yo..parachute
1FA83..1FA86 ; Emoji_Presentation # E13.0 [4] (🪃..🪆) boomerang..nesting dolls
1FA90..1FA95 ; Emoji_Presentation # E12.1 [6] (🪐..🪕) ringed planet..banjo
1FA96..1FAA8 ; Emoji_Presentation # E13.0 [19] (🪖..🪨) military helmet..rock
1FAB0..1FAB6 ; Emoji_Presentation # E13.0 [7] (🪰..🪶) fly..feather
1FAC0..1FAC2 ; Emoji_Presentation # E13.0 [3] (🫀..🫂) anatomical heart..people hugging
1FAD0..1FAD6 ; Emoji_Presentation # E13.0 [7] (🫐..🫖) blueberries..teapot
# Total elements: 1148
# ================================================
# All omitted code points have Emoji_Modifier=No
# @missing: 0000..10FFFF ; Emoji_Modifier ; No
1F3FB..1F3FF ; Emoji_Modifier # E2.0 [5] (🏻..🏿) light skin tone..dark skin tone
# Total elements: 5
# ================================================
# All omitted code points have Emoji_Modifier_Base=No
# @missing: 0000..10FFFF ; Emoji_Modifier_Base ; No
261D ; Emoji_Modifier_Base # E2.0 [1] (☝️) index pointing up
26F9 ; Emoji_Modifier_Base # E2.0 [1] (⛹️) person bouncing ball
270A..270D ; Emoji_Modifier_Base # E2.0 [4] (✊..✍️) raised fist..writing hand
1F385 ; Emoji_Modifier_Base # E2.0 [1] (🎅) Santa Claus
1F3C2..1F3C4 ; Emoji_Modifier_Base # E2.0 [3] (🏂..🏄) snowboarder..person surfing
1F3C7 ; Emoji_Modifier_Base # E2.0 [1] (🏇) horse racing
1F3CA..1F3CC ; Emoji_Modifier_Base # E2.0 [3] (🏊..🏌️) person swimming..person golfing
1F442..1F443 ; Emoji_Modifier_Base # E2.0 [2] (👂..👃) ear..nose
1F446..1F450 ; Emoji_Modifier_Base # E2.0 [11] (👆..👐) backhand index pointing up..open hands
1F466..1F478 ; Emoji_Modifier_Base # E2.0 [19] (👦..👸) boy..princess
1F47C ; Emoji_Modifier_Base # E2.0 [1] (👼) baby angel
1F481..1F483 ; Emoji_Modifier_Base # E2.0 [3] (💁..💃) person tipping hand..woman dancing
1F485..1F487 ; Emoji_Modifier_Base # E2.0 [3] (💅..💇) nail polish..person getting haircut
1F48F ; Emoji_Modifier_Base # E2.0 [1] (💏) kiss
1F491 ; Emoji_Modifier_Base # E2.0 [1] (💑) couple with heart
1F4AA ; Emoji_Modifier_Base # E2.0 [1] (💪) flexed biceps
1F574..1F575 ; Emoji_Modifier_Base # E2.0 [2] (🕴️..🕵️) man in suit levitating..detective
1F57A ; Emoji_Modifier_Base # E4.0 [1] (🕺) man dancing
1F590 ; Emoji_Modifier_Base # E2.0 [1] (🖐️) hand with fingers splayed
1F595..1F596 ; Emoji_Modifier_Base # E2.0 [2] (🖕..🖖) middle finger..vulcan salute
1F645..1F647 ; Emoji_Modifier_Base # E2.0 [3] (🙅..🙇) person gesturing NO..person bowing
1F64B..1F64F ; Emoji_Modifier_Base # E2.0 [5] (🙋..🙏) person raising hand..folded hands
1F6A3 ; Emoji_Modifier_Base # E2.0 [1] (🚣) person rowing boat
1F6B4..1F6B6 ; Emoji_Modifier_Base # E2.0 [3] (🚴..🚶) person biking..person walking
1F6C0 ; Emoji_Modifier_Base # E2.0 [1] (🛀) person taking bath
1F6CC ; Emoji_Modifier_Base # E2.0 [1] (🛌) person in bed
1F90C ; Emoji_Modifier_Base # E13.0 [1] (🤌) pinched fingers
1F90F ; Emoji_Modifier_Base # E12.1 [1] (🤏) pinching hand
1F918 ; Emoji_Modifier_Base # E2.0 [1] (🤘) sign of the horns
1F919..1F91E ; Emoji_Modifier_Base # E4.0 [6] (🤙..🤞) call me hand..crossed fingers
1F91F ; Emoji_Modifier_Base # E5.0 [1] (🤟) love-you gesture
1F926 ; Emoji_Modifier_Base # E4.0 [1] (🤦) person facepalming
1F930 ; Emoji_Modifier_Base # E4.0 [1] (🤰) pregnant woman
1F931..1F932 ; Emoji_Modifier_Base # E5.0 [2] (🤱..🤲) breast-feeding..palms up together
1F933..1F939 ; Emoji_Modifier_Base # E4.0 [7] (🤳..🤹) selfie..person juggling
1F93C..1F93E ; Emoji_Modifier_Base # E4.0 [3] (🤼..🤾) people wrestling..person playing handball
1F9B5..1F9B6 ; Emoji_Modifier_Base # E11.0 [2] (🦵..🦶) leg..foot
1F9B8..1F9B9 ; Emoji_Modifier_Base # E11.0 [2] (🦸..🦹) superhero..supervillain
1F9BB ; Emoji_Modifier_Base # E12.1 [1] (🦻) ear with hearing aid
1F9CD..1F9CF ; Emoji_Modifier_Base # E12.1 [3] (🧍..🧏) person standing..deaf person
1F9D1..1F9DD ; Emoji_Modifier_Base # E5.0 [13] (🧑..🧝) person..elf
# Total elements: 121
# ================================================
# All omitted code points have Emoji_Component=No
# @missing: 0000..10FFFF ; Emoji_Component ; No
0023 ; Emoji_Component # E0.0 [1] (#️) number sign
002A ; Emoji_Component # E0.0 [1] (*️) asterisk
0030..0039 ; Emoji_Component # E0.0 [10] (0️..9️) digit zero..digit nine
200D ; Emoji_Component # E0.0 [1] (‍) zero width joiner
20E3 ; Emoji_Component # E0.0 [1] (⃣) combining enclosing keycap
FE0F ; Emoji_Component # E0.0 [1] () VARIATION SELECTOR-16
1F1E6..1F1FF ; Emoji_Component # E0.0 [26] (🇦..🇿) regional indicator symbol letter a..regional indicator symbol letter z
1F3FB..1F3FF ; Emoji_Component # E2.0 [5] (🏻..🏿) light skin tone..dark skin tone
1F9B0..1F9B3 ; Emoji_Component # E11.0 [4] (🦰..🦳) red hair..white hair
E0020..E007F ; Emoji_Component # E0.0 [96] (󠀠..󠁿) tag space..cancel tag
# Total elements: 146
# ================================================
# All omitted code points have Extended_Pictographic=No
# @missing: 0000..10FFFF ; Extended_Pictographic ; No
00A9 ; Extended_Pictographic# E2.0 [1] (©️) copyright
00AE ; Extended_Pictographic# E2.0 [1] (®️) registered
203C ; Extended_Pictographic# E2.0 [1] (‼️) double exclamation mark
2049 ; Extended_Pictographic# E2.0 [1] (⁉️) exclamation question mark
2122 ; Extended_Pictographic# E2.0 [1] (™️) trade mark
2139 ; Extended_Pictographic# E2.0 [1] (ℹ️) information
2194..2199 ; Extended_Pictographic# E2.0 [6] (↔️..↙️) left-right arrow..down-left arrow
21A9..21AA ; Extended_Pictographic# E2.0 [2] (↩️..↪️) right arrow curving left..left arrow curving right
231A..231B ; Extended_Pictographic# E2.0 [2] (⌚..⌛) watch..hourglass done
2328 ; Extended_Pictographic# E2.0 [1] (⌨️) keyboard
2388 ; Extended_Pictographic# E0.0 [1] (⎈) HELM SYMBOL
23CF ; Extended_Pictographic# E2.0 [1] (⏏️) eject button
23E9..23F3 ; Extended_Pictographic# E2.0 [11] (⏩..⏳) fast-forward button..hourglass not done
23F8..23FA ; Extended_Pictographic# E2.0 [3] (⏸️..⏺️) pause button..record button
24C2 ; Extended_Pictographic# E2.0 [1] (Ⓜ️) circled M
25AA..25AB ; Extended_Pictographic# E2.0 [2] (▪️..▫️) black small square..white small square
25B6 ; Extended_Pictographic# E2.0 [1] (▶️) play button
25C0 ; Extended_Pictographic# E2.0 [1] (◀️) reverse button
25FB..25FE ; Extended_Pictographic# E2.0 [4] (◻️..◾) white medium square..black medium-small square
2600..2604 ; Extended_Pictographic# E2.0 [5] (☀️..☄️) sun..comet
2605 ; Extended_Pictographic# E0.0 [1] (★) BLACK STAR
2607..260D ; Extended_Pictographic# E0.0 [7] (☇..☍) LIGHTNING..OPPOSITION
260E ; Extended_Pictographic# E2.0 [1] (☎️) telephone
260F..2610 ; Extended_Pictographic# E0.0 [2] (☏..☐) WHITE TELEPHONE..BALLOT BOX
2611 ; Extended_Pictographic# E2.0 [1] (☑️) check box with check
2612 ; Extended_Pictographic# E0.0 [1] (☒) BALLOT BOX WITH X
2614..2615 ; Extended_Pictographic# E2.0 [2] (☔..☕) umbrella with rain drops..hot beverage
2616..2617 ; Extended_Pictographic# E0.0 [2] (☖..☗) WHITE SHOGI PIECE..BLACK SHOGI PIECE
2618 ; Extended_Pictographic# E2.0 [1] (☘️) shamrock
2619..261C ; Extended_Pictographic# E0.0 [4] (☙..☜) REVERSED ROTATED FLORAL HEART BULLET..WHITE LEFT POINTING INDEX
261D ; Extended_Pictographic# E2.0 [1] (☝️) index pointing up
261E..261F ; Extended_Pictographic# E0.0 [2] (☞..☟) WHITE RIGHT POINTING INDEX..WHITE DOWN POINTING INDEX
2620 ; Extended_Pictographic# E2.0 [1] (☠️) skull and crossbones
2621 ; Extended_Pictographic# E0.0 [1] (☡) CAUTION SIGN
2622..2623 ; Extended_Pictographic# E2.0 [2] (☢️..☣️) radioactive..biohazard
2624..2625 ; Extended_Pictographic# E0.0 [2] (☤..☥) CADUCEUS..ANKH
2626 ; Extended_Pictographic# E2.0 [1] (☦️) orthodox cross
2627..2629 ; Extended_Pictographic# E0.0 [3] (☧..☩) CHI RHO..CROSS OF JERUSALEM
262A ; Extended_Pictographic# E2.0 [1] (☪️) star and crescent
262B..262D ; Extended_Pictographic# E0.0 [3] (☫..☭) FARSI SYMBOL..HAMMER AND SICKLE
262E..262F ; Extended_Pictographic# E2.0 [2] (☮️..☯️) peace symbol..yin yang
2630..2637 ; Extended_Pictographic# E0.0 [8] (☰..☷) TRIGRAM FOR HEAVEN..TRIGRAM FOR EARTH
2638..263A ; Extended_Pictographic# E2.0 [3] (☸️..☺️) wheel of dharma..smiling face
263B..263F ; Extended_Pictographic# E0.0 [5] (☻..☿) BLACK SMILING FACE..MERCURY
2640 ; Extended_Pictographic# E4.0 [1] (♀️) female sign
2641 ; Extended_Pictographic# E0.0 [1] (♁) EARTH
2642 ; Extended_Pictographic# E4.0 [1] (♂️) male sign
2643..2647 ; Extended_Pictographic# E0.0 [5] (♃..♇) JUPITER..PLUTO
2648..2653 ; Extended_Pictographic# E2.0 [12] (♈..♓) Aries..Pisces
2654..265E ; Extended_Pictographic# E0.0 [11] (♔..♞) WHITE CHESS KING..BLACK CHESS KNIGHT
265F ; Extended_Pictographic# E11.0 [1] (♟️) chess pawn
2660 ; Extended_Pictographic# E2.0 [1] (♠️) spade suit
2661..2662 ; Extended_Pictographic# E0.0 [2] (♡..♢) WHITE HEART SUIT..WHITE DIAMOND SUIT
2663 ; Extended_Pictographic# E2.0 [1] (♣️) club suit
2664 ; Extended_Pictographic# E0.0 [1] (♤) WHITE SPADE SUIT
2665..2666 ; Extended_Pictographic# E2.0 [2] (♥️..♦️) heart suit..diamond suit
2667 ; Extended_Pictographic# E0.0 [1] (♧) WHITE CLUB SUIT
2668 ; Extended_Pictographic# E2.0 [1] (♨️) hot springs
2669..267A ; Extended_Pictographic# E0.0 [18] (♩..♺) QUARTER NOTE..RECYCLING SYMBOL FOR GENERIC MATERIALS
267B ; Extended_Pictographic# E2.0 [1] (♻️) recycling symbol
267C..267D ; Extended_Pictographic# E0.0 [2] (♼..♽) RECYCLED PAPER SYMBOL..PARTIALLY-RECYCLED PAPER SYMBOL
267E ; Extended_Pictographic# E11.0 [1] (♾️) infinity
267F ; Extended_Pictographic# E2.0 [1] (♿) wheelchair symbol
2680..2685 ; Extended_Pictographic# E0.0 [6] (⚀..⚅) DIE FACE-1..DIE FACE-6
2690..2691 ; Extended_Pictographic# E0.0 [2] (⚐..⚑) WHITE FLAG..BLACK FLAG
2692..2694 ; Extended_Pictographic# E2.0 [3] (⚒️..⚔️) hammer and pick..crossed swords
2695 ; Extended_Pictographic# E4.0 [1] (⚕️) medical symbol
2696..2697 ; Extended_Pictographic# E2.0 [2] (⚖️..⚗️) balance scale..alembic
2698 ; Extended_Pictographic# E0.0 [1] (⚘) FLOWER
2699 ; Extended_Pictographic# E2.0 [1] (⚙️) gear
269A ; Extended_Pictographic# E0.0 [1] (⚚) STAFF OF HERMES
269B..269C ; Extended_Pictographic# E2.0 [2] (⚛️..⚜️) atom symbol..fleur-de-lis
269D..269F ; Extended_Pictographic# E0.0 [3] (⚝..⚟) OUTLINED WHITE STAR..THREE LINES CONVERGING LEFT
26A0..26A1 ; Extended_Pictographic# E2.0 [2] (⚠️..⚡) warning..high voltage
26A2..26A6 ; Extended_Pictographic# E0.0 [5] (⚢..⚦) DOUBLED FEMALE SIGN..MALE WITH STROKE SIGN
26A7 ; Extended_Pictographic# E13.0 [1] (⚧️) transgender symbol
26A8..26A9 ; Extended_Pictographic# E0.0 [2] (⚨..⚩) VERTICAL MALE WITH STROKE SIGN..HORIZONTAL MALE WITH STROKE SIGN
26AA..26AB ; Extended_Pictographic# E2.0 [2] (⚪..⚫) white circle..black circle
26AC..26AF ; Extended_Pictographic# E0.0 [4] (⚬..⚯) MEDIUM SMALL WHITE CIRCLE..UNMARRIED PARTNERSHIP SYMBOL
26B0..26B1 ; Extended_Pictographic# E2.0 [2] (⚰️..⚱️) coffin..funeral urn
26B2..26BC ; Extended_Pictographic# E0.0 [11] (⚲..⚼) NEUTER..SESQUIQUADRATE
26BD..26BE ; Extended_Pictographic# E2.0 [2] (⚽..⚾) soccer ball..baseball
26BF..26C3 ; Extended_Pictographic# E0.0 [5] (⚿..⛃) SQUARED KEY..BLACK DRAUGHTS KING
26C4..26C5 ; Extended_Pictographic# E2.0 [2] (⛄..⛅) snowman without snow..sun behind cloud
26C6..26C7 ; Extended_Pictographic# E0.0 [2] (⛆..⛇) RAIN..BLACK SNOWMAN
26C8 ; Extended_Pictographic# E2.0 [1] (⛈️) cloud with lightning and rain
26C9..26CD ; Extended_Pictographic# E0.0 [5] (⛉..⛍) TURNED WHITE SHOGI PIECE..DISABLED CAR
26CE..26CF ; Extended_Pictographic# E2.0 [2] (⛎..⛏️) Ophiuchus..pick
26D0 ; Extended_Pictographic# E0.0 [1] (⛐) CAR SLIDING
26D1 ; Extended_Pictographic# E2.0 [1] (⛑️) rescue worker’s helmet
26D2 ; Extended_Pictographic# E0.0 [1] (⛒) CIRCLED CROSSING LANES
26D3..26D4 ; Extended_Pictographic# E2.0 [2] (⛓️..⛔) chains..no entry
26D5..26E8 ; Extended_Pictographic# E0.0 [20] (⛕..⛨) ALTERNATE ONE-WAY LEFT WAY TRAFFIC..BLACK CROSS ON SHIELD
26E9..26EA ; Extended_Pictographic# E2.0 [2] (⛩️..⛪) shinto shrine..church
26EB..26EF ; Extended_Pictographic# E0.0 [5] (⛫..⛯) CASTLE..MAP SYMBOL FOR LIGHTHOUSE
26F0..26F5 ; Extended_Pictographic# E2.0 [6] (⛰️..⛵) mountain..sailboat
26F6 ; Extended_Pictographic# E0.0 [1] (⛶) SQUARE FOUR CORNERS
26F7..26FA ; Extended_Pictographic# E2.0 [4] (⛷️..⛺) skier..tent
26FB..26FC ; Extended_Pictographic# E0.0 [2] (⛻..⛼) JAPANESE BANK SYMBOL..HEADSTONE GRAVEYARD SYMBOL
26FD ; Extended_Pictographic# E2.0 [1] (⛽) fuel pump
26FE..2701 ; Extended_Pictographic# E0.0 [4] (⛾..✁) CUP ON BLACK SQUARE..UPPER BLADE SCISSORS
2702 ; Extended_Pictographic# E2.0 [1] (✂️) scissors
2703..2704 ; Extended_Pictographic# E0.0 [2] (✃..✄) LOWER BLADE SCISSORS..WHITE SCISSORS
2705 ; Extended_Pictographic# E2.0 [1] (✅) check mark button
2708..270D ; Extended_Pictographic# E2.0 [6] (✈️..✍️) airplane..writing hand
270E ; Extended_Pictographic# E0.0 [1] (✎) LOWER RIGHT PENCIL
270F ; Extended_Pictographic# E2.0 [1] (✏️) pencil
2710..2711 ; Extended_Pictographic# E0.0 [2] (✐..✑) UPPER RIGHT PENCIL..WHITE NIB
2712 ; Extended_Pictographic# E2.0 [1] (✒️) black nib
2714 ; Extended_Pictographic# E2.0 [1] (✔️) check mark
2716 ; Extended_Pictographic# E2.0 [1] (✖️) multiplication sign
271D ; Extended_Pictographic# E2.0 [1] (✝️) latin cross
2721 ; Extended_Pictographic# E2.0 [1] (✡️) star of David
2728 ; Extended_Pictographic# E2.0 [1] (✨) sparkles
2733..2734 ; Extended_Pictographic# E2.0 [2] (✳️..✴️) eight-spoked asterisk..eight-pointed star
2744 ; Extended_Pictographic# E2.0 [1] (❄️) snowflake
2747 ; Extended_Pictographic# E2.0 [1] (❇️) sparkle
274C ; Extended_Pictographic# E2.0 [1] (❌) cross mark
274E ; Extended_Pictographic# E2.0 [1] (❎) cross mark button
2753..2755 ; Extended_Pictographic# E2.0 [3] (❓..❕) question mark..white exclamation mark
2757 ; Extended_Pictographic# E2.0 [1] (❗) exclamation mark
2763..2764 ; Extended_Pictographic# E2.0 [2] (❣️..❤️) heart exclamation..red heart
2765..2767 ; Extended_Pictographic# E0.0 [3] (❥..❧) ROTATED HEAVY BLACK HEART BULLET..ROTATED FLORAL HEART BULLET
2795..2797 ; Extended_Pictographic# E2.0 [3] (➕..➗) plus sign..division sign
27A1 ; Extended_Pictographic# E2.0 [1] (➡️) right arrow
27B0 ; Extended_Pictographic# E2.0 [1] (➰) curly loop
27BF ; Extended_Pictographic# E2.0 [1] (➿) double curly loop
2934..2935 ; Extended_Pictographic# E2.0 [2] (⤴️..⤵️) right arrow curving up..right arrow curving down
2B05..2B07 ; Extended_Pictographic# E2.0 [3] (⬅️..⬇️) left arrow..down arrow
2B1B..2B1C ; Extended_Pictographic# E2.0 [2] (⬛..⬜) black large square..white large square
2B50 ; Extended_Pictographic# E2.0 [1] (⭐) star
2B55 ; Extended_Pictographic# E2.0 [1] (⭕) hollow red circle
3030 ; Extended_Pictographic# E2.0 [1] (〰️) wavy dash
303D ; Extended_Pictographic# E2.0 [1] (〽️) part alternation mark
3297 ; Extended_Pictographic# E2.0 [1] (㊗️) Japanese “congratulations” button
3299 ; Extended_Pictographic# E2.0 [1] (㊙️) Japanese “secret” button
1F000..1F003 ; Extended_Pictographic# E0.0 [4] (🀀..🀃) MAHJONG TILE EAST WIND..MAHJONG TILE NORTH WIND
1F004 ; Extended_Pictographic# E2.0 [1] (🀄) mahjong red dragon
1F005..1F0CE ; Extended_Pictographic# E0.0 [202] (🀅..🃎) MAHJONG TILE GREEN DRAGON..PLAYING CARD KING OF DIAMONDS
1F0CF ; Extended_Pictographic# E2.0 [1] (🃏) joker
1F0D0..1F0FF ; Extended_Pictographic# E0.0 [48] (🃐..🃿) <reserved-1F0D0>..<reserved-1F0FF>
1F10D..1F10F ; Extended_Pictographic# E0.0 [3] (🄍..🄏) CIRCLED ZERO WITH SLASH..CIRCLED DOLLAR SIGN WITH OVERLAID BACKSLASH
1F12F ; Extended_Pictographic# E0.0 [1] (🄯) COPYLEFT SYMBOL
1F16C..1F16F ; Extended_Pictographic# E0.0 [4] (🅬..🅯) RAISED MR SIGN..CIRCLED HUMAN FIGURE
1F170..1F171 ; Extended_Pictographic# E2.0 [2] (🅰️..🅱️) A button (blood type)..B button (blood type)
1F17E..1F17F ; Extended_Pictographic# E2.0 [2] (🅾️..🅿️) O button (blood type)..P button
1F18E ; Extended_Pictographic# E2.0 [1] (🆎) AB button (blood type)
1F191..1F19A ; Extended_Pictographic# E2.0 [10] (🆑..🆚) CL button..VS button
1F1AD..1F1E5 ; Extended_Pictographic# E0.0 [57] (🆭..🇥) MASK WORK SYMBOL..<reserved-1F1E5>
1F201..1F202 ; Extended_Pictographic# E2.0 [2] (🈁..🈂️) Japanese “here” button..Japanese “service charge” button
1F203..1F20F ; Extended_Pictographic# E0.0 [13] (🈃..🈏) <reserved-1F203>..<reserved-1F20F>
1F21A ; Extended_Pictographic# E2.0 [1] (🈚) Japanese “free of charge” button
1F22F ; Extended_Pictographic# E2.0 [1] (🈯) Japanese “reserved” button
1F232..1F23A ; Extended_Pictographic# E2.0 [9] (🈲..🈺) Japanese “prohibited” button..Japanese “open for business” button
1F23C..1F23F ; Extended_Pictographic# E0.0 [4] (🈼..🈿) <reserved-1F23C>..<reserved-1F23F>
1F249..1F24F ; Extended_Pictographic# E0.0 [7] (🉉..🉏) <reserved-1F249>..<reserved-1F24F>
1F250..1F251 ; Extended_Pictographic# E2.0 [2] (🉐..🉑) Japanese “bargain” button..Japanese “acceptable” button
1F252..1F2FF ; Extended_Pictographic# E0.0 [174] (🉒..🋿) <reserved-1F252>..<reserved-1F2FF>
1F300..1F321 ; Extended_Pictographic# E2.0 [34] (🌀..🌡️) cyclone..thermometer
1F322..1F323 ; Extended_Pictographic# E0.0 [2] (🌢..🌣) BLACK DROPLET..WHITE SUN
1F324..1F393 ; Extended_Pictographic# E2.0 [112] (🌤️..🎓) sun behind small cloud..graduation cap
1F394..1F395 ; Extended_Pictographic# E0.0 [2] (🎔..🎕) HEART WITH TIP ON THE LEFT..BOUQUET OF FLOWERS
1F396..1F397 ; Extended_Pictographic# E2.0 [2] (🎖️..🎗️) military medal..reminder ribbon
1F398 ; Extended_Pictographic# E0.0 [1] (🎘) MUSICAL KEYBOARD WITH JACKS
1F399..1F39B ; Extended_Pictographic# E2.0 [3] (🎙️..🎛️) studio microphone..control knobs
1F39C..1F39D ; Extended_Pictographic# E0.0 [2] (🎜..🎝) BEAMED ASCENDING MUSICAL NOTES..BEAMED DESCENDING MUSICAL NOTES
1F39E..1F3F0 ; Extended_Pictographic# E2.0 [83] (🎞️..🏰) film frames..castle
1F3F1..1F3F2 ; Extended_Pictographic# E0.0 [2] (🏱..🏲) WHITE PENNANT..BLACK PENNANT
1F3F3..1F3F5 ; Extended_Pictographic# E2.0 [3] (🏳️..🏵️) white flag..rosette
1F3F6 ; Extended_Pictographic# E0.0 [1] (🏶) BLACK ROSETTE
1F3F7..1F3FA ; Extended_Pictographic# E2.0 [4] (🏷️..🏺) label..amphora
1F400..1F4FD ; Extended_Pictographic# E2.0 [254] (🐀..📽️) rat..film projector
1F4FE ; Extended_Pictographic# E0.0 [1] (📾) PORTABLE STEREO
1F4FF..1F53D ; Extended_Pictographic# E2.0 [63] (📿..🔽) prayer beads..downwards button
1F546..1F548 ; Extended_Pictographic# E0.0 [3] (🕆..🕈) WHITE LATIN CROSS..CELTIC CROSS
1F549..1F54E ; Extended_Pictographic# E2.0 [6] (🕉️..🕎) om..menorah
1F54F ; Extended_Pictographic# E0.0 [1] (🕏) BOWL OF HYGIEIA
1F550..1F567 ; Extended_Pictographic# E2.0 [24] (🕐..🕧) one o’clock..twelve-thirty
1F568..1F56E ; Extended_Pictographic# E0.0 [7] (🕨..🕮) RIGHT SPEAKER..BOOK
1F56F..1F570 ; Extended_Pictographic# E2.0 [2] (🕯️..🕰️) candle..mantelpiece clock
1F571..1F572 ; Extended_Pictographic# E0.0 [2] (🕱..🕲) BLACK SKULL AND CROSSBONES..NO PIRACY
1F573..1F579 ; Extended_Pictographic# E2.0 [7] (🕳️..🕹️) hole..joystick
1F57A ; Extended_Pictographic# E4.0 [1] (🕺) man dancing
1F57B..1F586 ; Extended_Pictographic# E0.0 [12] (🕻..🖆) LEFT HAND TELEPHONE RECEIVER..PEN OVER STAMPED ENVELOPE
1F587 ; Extended_Pictographic# E2.0 [1] (🖇️) linked paperclips
1F588..1F589 ; Extended_Pictographic# E0.0 [2] (🖈..🖉) BLACK PUSHPIN..LOWER LEFT PENCIL
1F58A..1F58D ; Extended_Pictographic# E2.0 [4] (🖊️..🖍️) pen..crayon
1F58E..1F58F ; Extended_Pictographic# E0.0 [2] (🖎..🖏) LEFT WRITING HAND..TURNED OK HAND SIGN
1F590 ; Extended_Pictographic# E2.0 [1] (🖐️) hand with fingers splayed
1F591..1F594 ; Extended_Pictographic# E0.0 [4] (🖑..🖔) REVERSED RAISED HAND WITH FINGERS SPLAYED..REVERSED VICTORY HAND
1F595..1F596 ; Extended_Pictographic# E2.0 [2] (🖕..🖖) middle finger..vulcan salute
1F597..1F5A3 ; Extended_Pictographic# E0.0 [13] (🖗..🖣) WHITE DOWN POINTING LEFT HAND INDEX..BLACK DOWN POINTING BACKHAND INDEX
1F5A4 ; Extended_Pictographic# E4.0 [1] (🖤) black heart
1F5A5 ; Extended_Pictographic# E2.0 [1] (🖥️) desktop computer
1F5A6..1F5A7 ; Extended_Pictographic# E0.0 [2] (🖦..🖧) KEYBOARD AND MOUSE..THREE NETWORKED COMPUTERS
1F5A8 ; Extended_Pictographic# E2.0 [1] (🖨️) printer
1F5A9..1F5B0 ; Extended_Pictographic# E0.0 [8] (🖩..🖰) POCKET CALCULATOR..TWO BUTTON MOUSE
1F5B1..1F5B2 ; Extended_Pictographic# E2.0 [2] (🖱️..🖲️) computer mouse..trackball
1F5B3..1F5BB ; Extended_Pictographic# E0.0 [9] (🖳..🖻) OLD PERSONAL COMPUTER..DOCUMENT WITH PICTURE
1F5BC ; Extended_Pictographic# E2.0 [1] (🖼️) framed picture
1F5BD..1F5C1 ; Extended_Pictographic# E0.0 [5] (🖽..🗁) FRAME WITH TILES..OPEN FOLDER
1F5C2..1F5C4 ; Extended_Pictographic# E2.0 [3] (🗂️..🗄️) card index dividers..file cabinet
1F5C5..1F5D0 ; Extended_Pictographic# E0.0 [12] (🗅..🗐) EMPTY NOTE..PAGES
1F5D1..1F5D3 ; Extended_Pictographic# E2.0 [3] (🗑️..🗓️) wastebasket..spiral calendar
1F5D4..1F5DB ; Extended_Pictographic# E0.0 [8] (🗔..🗛) DESKTOP WINDOW..DECREASE FONT SIZE SYMBOL
1F5DC..1F5DE ; Extended_Pictographic# E2.0 [3] (🗜️..🗞️) clamp..rolled-up newspaper
1F5DF..1F5E0 ; Extended_Pictographic# E0.0 [2] (🗟..🗠) PAGE WITH CIRCLED TEXT..STOCK CHART
1F5E1 ; Extended_Pictographic# E2.0 [1] (🗡️) dagger
1F5E2 ; Extended_Pictographic# E0.0 [1] (🗢) LIPS
1F5E3 ; Extended_Pictographic# E2.0 [1] (🗣️) speaking head
1F5E4..1F5E7 ; Extended_Pictographic# E0.0 [4] (🗤..🗧) THREE RAYS ABOVE..THREE RAYS RIGHT
1F5E8 ; Extended_Pictographic# E2.0 [1] (🗨️) left speech bubble
1F5E9..1F5EE ; Extended_Pictographic# E0.0 [6] (🗩..🗮) RIGHT SPEECH BUBBLE..LEFT ANGER BUBBLE
1F5EF ; Extended_Pictographic# E2.0 [1] (🗯️) right anger bubble
1F5F0..1F5F2 ; Extended_Pictographic# E0.0 [3] (🗰..🗲) MOOD BUBBLE..LIGHTNING MOOD
1F5F3 ; Extended_Pictographic# E2.0 [1] (🗳️) ballot box with ballot
1F5F4..1F5F9 ; Extended_Pictographic# E0.0 [6] (🗴..🗹) BALLOT SCRIPT X..BALLOT BOX WITH BOLD CHECK
1F5FA..1F64F ; Extended_Pictographic# E2.0 [86] (🗺️..🙏) world map..folded hands
1F680..1F6C5 ; Extended_Pictographic# E2.0 [70] (🚀..🛅) rocket..left luggage
1F6C6..1F6CA ; Extended_Pictographic# E0.0 [5] (🛆..🛊) TRIANGLE WITH ROUNDED CORNERS..GIRLS SYMBOL
1F6CB..1F6D0 ; Extended_Pictographic# E2.0 [6] (🛋️..🛐) couch and lamp..place of worship
1F6D1..1F6D2 ; Extended_Pictographic# E4.0 [2] (🛑..🛒) stop sign..shopping cart
1F6D3..1F6D4 ; Extended_Pictographic# E0.0 [2] (🛓..🛔) STUPA..PAGODA
1F6D5 ; Extended_Pictographic# E12.1 [1] (🛕) hindu temple
1F6D6..1F6D7 ; Extended_Pictographic# E13.0 [2] (🛖..🛗) hut..elevator
1F6D8..1F6DF ; Extended_Pictographic# E0.0 [8] (🛘..🛟) <reserved-1F6D8>..<reserved-1F6DF>
1F6E0..1F6E5 ; Extended_Pictographic# E2.0 [6] (🛠️..🛥️) hammer and wrench..motor boat
1F6E6..1F6E8 ; Extended_Pictographic# E0.0 [3] (🛦..🛨) UP-POINTING MILITARY AIRPLANE..UP-POINTING SMALL AIRPLANE
1F6E9 ; Extended_Pictographic# E2.0 [1] (🛩️) small airplane
1F6EA ; Extended_Pictographic# E0.0 [1] (🛪) NORTHEAST-POINTING AIRPLANE
1F6EB..1F6EC ; Extended_Pictographic# E2.0 [2] (🛫..🛬) airplane departure..airplane arrival
1F6ED..1F6EF ; Extended_Pictographic# E0.0 [3] (🛭..🛯) <reserved-1F6ED>..<reserved-1F6EF>
1F6F0 ; Extended_Pictographic# E2.0 [1] (🛰️) satellite
1F6F1..1F6F2 ; Extended_Pictographic# E0.0 [2] (🛱..🛲) ONCOMING FIRE ENGINE..DIESEL LOCOMOTIVE
1F6F3 ; Extended_Pictographic# E2.0 [1] (🛳️) passenger ship
1F6F4..1F6F6 ; Extended_Pictographic# E4.0 [3] (🛴..🛶) kick scooter..canoe
1F6F7..1F6F8 ; Extended_Pictographic# E5.0 [2] (🛷..🛸) sled..flying saucer
1F6F9 ; Extended_Pictographic# E11.0 [1] (🛹) skateboard
1F6FA ; Extended_Pictographic# E12.1 [1] (🛺) auto rickshaw
1F6FB..1F6FC ; Extended_Pictographic# E13.0 [2] (🛻..🛼) pickup truck..roller skate
1F6FD..1F6FF ; Extended_Pictographic# E0.0 [3] (🛽..🛿) <reserved-1F6FD>..<reserved-1F6FF>
1F774..1F77F ; Extended_Pictographic# E0.0 [12] (🝴..🝿) <reserved-1F774>..<reserved-1F77F>
1F7D5..1F7DF ; Extended_Pictographic# E0.0 [11] (🟕..🟟) CIRCLED TRIANGLE..<reserved-1F7DF>
1F7E0..1F7EB ; Extended_Pictographic# E12.1 [12] (🟠..🟫) orange circle..brown square
1F7EC..1F7FF ; Extended_Pictographic# E0.0 [20] (🟬..🟿) <reserved-1F7EC>..<reserved-1F7FF>
1F80C..1F80F ; Extended_Pictographic# E0.0 [4] (🠌..🠏) <reserved-1F80C>..<reserved-1F80F>
1F848..1F84F ; Extended_Pictographic# E0.0 [8] (🡈..🡏) <reserved-1F848>..<reserved-1F84F>
1F85A..1F85F ; Extended_Pictographic# E0.0 [6] (🡚..🡟) <reserved-1F85A>..<reserved-1F85F>
1F888..1F88F ; Extended_Pictographic# E0.0 [8] (🢈..🢏) <reserved-1F888>..<reserved-1F88F>
1F8AE..1F8FF ; Extended_Pictographic# E0.0 [82] (🢮..🣿) <reserved-1F8AE>..<reserved-1F8FF>
1F90C ; Extended_Pictographic# E13.0 [1] (🤌) pinched fingers
1F90D..1F90F ; Extended_Pictographic# E12.1 [3] (🤍..🤏) white heart..pinching hand
1F910..1F918 ; Extended_Pictographic# E2.0 [9] (🤐..🤘) zipper-mouth face..sign of the horns
1F919..1F91E ; Extended_Pictographic# E4.0 [6] (🤙..🤞) call me hand..crossed fingers
1F91F ; Extended_Pictographic# E5.0 [1] (🤟) love-you gesture
1F920..1F927 ; Extended_Pictographic# E4.0 [8] (🤠..🤧) cowboy hat face..sneezing face
1F928..1F92F ; Extended_Pictographic# E5.0 [8] (🤨..🤯) face with raised eyebrow..exploding head
1F930 ; Extended_Pictographic# E4.0 [1] (🤰) pregnant woman
1F931..1F932 ; Extended_Pictographic# E5.0 [2] (🤱..🤲) breast-feeding..palms up together
1F933..1F93A ; Extended_Pictographic# E4.0 [8] (🤳..🤺) selfie..person fencing
1F93C..1F93E ; Extended_Pictographic# E4.0 [3] (🤼..🤾) people wrestling..person playing handball
1F93F ; Extended_Pictographic# E12.1 [1] (🤿) diving mask
1F940..1F945 ; Extended_Pictographic# E4.0 [6] (🥀..🥅) wilted flower..goal net
1F947..1F94B ; Extended_Pictographic# E4.0 [5] (🥇..🥋) 1st place medal..martial arts uniform
1F94C ; Extended_Pictographic# E5.0 [1] (🥌) curling stone
1F94D..1F94F ; Extended_Pictographic# E11.0 [3] (🥍..🥏) lacrosse..flying disc
1F950..1F95E ; Extended_Pictographic# E4.0 [15] (🥐..🥞) croissant..pancakes
1F95F..1F96B ; Extended_Pictographic# E5.0 [13] (🥟..🥫) dumpling..canned food
1F96C..1F970 ; Extended_Pictographic# E11.0 [5] (🥬..🥰) leafy green..smiling face with hearts
1F971 ; Extended_Pictographic# E12.1 [1] (🥱) yawning face
1F972 ; Extended_Pictographic# E13.0 [1] (🥲) smiling face with tear
1F973..1F976 ; Extended_Pictographic# E11.0 [4] (🥳..🥶) partying face..cold face
1F977..1F978 ; Extended_Pictographic# E13.0 [2] (🥷..🥸) ninja..disguised face
1F979 ; Extended_Pictographic# E0.0 [1] (🥹) <reserved-1F979>
1F97A ; Extended_Pictographic# E11.0 [1] (🥺) pleading face
1F97B ; Extended_Pictographic# E12.1 [1] (🥻) sari
1F97C..1F97F ; Extended_Pictographic# E11.0 [4] (🥼..🥿) lab coat..flat shoe
1F980..1F984 ; Extended_Pictographic# E2.0 [5] (🦀..🦄) crab..unicorn
1F985..1F991 ; Extended_Pictographic# E4.0 [13] (🦅..🦑) eagle..squid
1F992..1F997 ; Extended_Pictographic# E5.0 [6] (🦒..🦗) giraffe..cricket
1F998..1F9A2 ; Extended_Pictographic# E11.0 [11] (🦘..🦢) kangaroo..swan
1F9A3..1F9A4 ; Extended_Pictographic# E13.0 [2] (🦣..🦤) mammoth..dodo
1F9A5..1F9AA ; Extended_Pictographic# E12.1 [6] (🦥..🦪) sloth..oyster
1F9AB..1F9AD ; Extended_Pictographic# E13.0 [3] (🦫..🦭) beaver..seal
1F9AE..1F9AF ; Extended_Pictographic# E12.1 [2] (🦮..🦯) guide dog..probing cane
1F9B0..1F9B9 ; Extended_Pictographic# E11.0 [10] (🦰..🦹) red hair..supervillain
1F9BA..1F9BF ; Extended_Pictographic# E12.1 [6] (🦺..🦿) safety vest..mechanical leg
1F9C0 ; Extended_Pictographic# E2.0 [1] (🧀) cheese wedge
1F9C1..1F9C2 ; Extended_Pictographic# E11.0 [2] (🧁..🧂) cupcake..salt
1F9C3..1F9CA ; Extended_Pictographic# E12.1 [8] (🧃..🧊) beverage box..ice
1F9CB ; Extended_Pictographic# E13.0 [1] (🧋) bubble tea
1F9CC ; Extended_Pictographic# E0.0 [1] (🧌) <reserved-1F9CC>
1F9CD..1F9CF ; Extended_Pictographic# E12.1 [3] (🧍..🧏) person standing..deaf person
1F9D0..1F9E6 ; Extended_Pictographic# E5.0 [23] (🧐..🧦) face with monocle..socks
1F9E7..1F9FF ; Extended_Pictographic# E11.0 [25] (🧧..🧿) red envelope..nazar amulet
1FA00..1FA6F ; Extended_Pictographic# E0.0 [112] (🨀..🩯) NEUTRAL CHESS KING..<reserved-1FA6F>
1FA70..1FA73 ; Extended_Pictographic# E12.1 [4] (🩰..🩳) ballet shoes..shorts
1FA74 ; Extended_Pictographic# E13.0 [1] (🩴) thong sandal
1FA75..1FA77 ; Extended_Pictographic# E0.0 [3] (🩵..🩷) <reserved-1FA75>..<reserved-1FA77>
1FA78..1FA7A ; Extended_Pictographic# E12.1 [3] (🩸..🩺) drop of blood..stethoscope
1FA7B..1FA7F ; Extended_Pictographic# E0.0 [5] (🩻..🩿) <reserved-1FA7B>..<reserved-1FA7F>
1FA80..1FA82 ; Extended_Pictographic# E12.1 [3] (🪀..🪂) yo-yo..parachute
1FA83..1FA86 ; Extended_Pictographic# E13.0 [4] (🪃..🪆) boomerang..nesting dolls
1FA87..1FA8F ; Extended_Pictographic# E0.0 [9] (🪇..🪏) <reserved-1FA87>..<reserved-1FA8F>
1FA90..1FA95 ; Extended_Pictographic# E12.1 [6] (🪐..🪕) ringed planet..banjo
1FA96..1FAA8 ; Extended_Pictographic# E13.0 [19] (🪖..🪨) military helmet..rock
1FAA9..1FAAF ; Extended_Pictographic# E0.0 [7] (🪩..🪯) <reserved-1FAA9>..<reserved-1FAAF>
1FAB0..1FAB6 ; Extended_Pictographic# E13.0 [7] (🪰..🪶) fly..feather
1FAB7..1FABF ; Extended_Pictographic# E0.0 [9] (🪷..🪿) <reserved-1FAB7>..<reserved-1FABF>
1FAC0..1FAC2 ; Extended_Pictographic# E13.0 [3] (🫀..🫂) anatomical heart..people hugging
1FAC3..1FACF ; Extended_Pictographic# E0.0 [13] (🫃..🫏) <reserved-1FAC3>..<reserved-1FACF>
1FAD0..1FAD6 ; Extended_Pictographic# E13.0 [7] (🫐..🫖) blueberries..teapot
1FAD7..1FFFD ; Extended_Pictographic# E0.0[1319] (🫗..🿽) <reserved-1FAD7>..<reserved-1FFFD>
# Total elements: 3793
#EOF
@@ -0,0 +1,36 @@
var http = require('http');
var fs = require('fs');
function fetchFile(PATH, DEST) {
var options = {
hostname: 'www.unicode.org',
port: 80,
path: PATH,
method: 'GET'
};
var req = http.request(options, (res) => {
var dest = fs.createWriteStream(DEST);
console.log(`STATUS: ${res.statusCode}`);
console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
res.pipe(dest);
});
req.on('error', (e) => {
console.log(`problem with request: ${e.message}`);
});
// write data to request body
req.end();
}
fetchFile(`/Public/13.0.0/ucd/UnicodeData-13.0.0d6.txt`, './UnicodeData.txt');
fetchFile(`/Public/emoji/12.1/emoji-test.txt`, './emoji-test.txt');
fetchFile(`/Public/13.0.0/ucd/auxiliary/GraphemeBreakProperty-13.0.0d4.txt`, './GraphemeBreakProperty.txt');
fetchFile(`/Public/13.0.0/ucd/emoji/emoji-data.txt`, './emoji-data.txt');
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,36 @@
var http = require('http');
var fs = require('fs');
function fetchFile(PATH, DEST) {
var options = {
hostname: 'www.unicode.org',
port: 80,
path: PATH,
method: 'GET'
};
var req = http.request(options, (res) => {
var dest = fs.createWriteStream(DEST);
console.log(`STATUS: ${res.statusCode}`);
console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
res.pipe(dest);
});
req.on('error', (e) => {
console.log(`problem with request: ${e.message}`);
});
// write data to request body
req.end();
}
fetchFile(`/Public/13.0.0/ucd/UnicodeData.txt`, './UnicodeData.txt');
fetchFile(`/Public/emoji/13.1/emoji-test.txt`, './emoji-test.txt');
fetchFile(`/Public/13.0.0/ucd/auxiliary/GraphemeBreakProperty.txt`, './GraphemeBreakProperty.txt');
fetchFile(`/Public/13.0.0/ucd/emoji/emoji-data.txt`, './emoji-data.txt');
@@ -0,0 +1,15 @@
# Unicode data embedded in Monaco Editor 0.30.1
The editor and base worker contain generated Unicode data through Microsoft VS Code's vs/base/common/strings.ts. Existing Microsoft notices remain applicable to the surrounding code.
The original alexdima/unicode-utils generators reproduce all four selected data expressions from fixed VS Code commit 829382514cb1065f5ebb90f436e1c6103e153953 and both original Monaco source maps:
- RTL regular expression: UnicodeData-10.0.0d5.txt (draft), frozen at 5c256860b1e4e54b0375165b8451cf7cdabc953a. The version label comes from that commit's fetch recipe; this is not a claim about final Unicode 10.
- Emoji regular expression and imprecise code-point predicate: Emoji 13.1 emoji-test.txt and Unicode 13.0 UnicodeData.txt, frozen at 6d37cf4362203bfb5431c560cd8809b2cc3b51c1.
- Grapheme break tree: 5,034 integers from GraphemeBreakProperty-13.0.0d4.txt and the accompanying 2019 emoji-data.txt snapshot, frozen at 2068fd0695b2798d8debb3646c860f5d37ba75b2. Do not silently substitute final or later Unicode data.
Data files retain their original Unicode copyright headers. LICENSE.txt is the complete Unicode data/software license from unicode-org/unicodetools commit b1c21ef7e972ea0c783c19cf25494bde645fbf21, copyright 1991-2020 Unicode, Inc.; it is a fixed historical reference, not a claim that the generator distributed its own license file.
The generator repository's package.json identifies Alex Dima and declares MIT; that frozen tree has no standalone LICENSE file. Its lockfile resolves regexpu 3.3.0 despite conflicting dependency/devDependency ranges. Reproduction tools use the exact locked dependencies in an isolated cache; those tools and dependencies are not shipped with the site runtime.
The source/input hashes, recipes, archive integrity and reproduction command are maintained in refactor/baselines/monaco-unicode-provenance.json and scripts/site-vendor/monaco/README.md in the ArtPlayer source repository. This records data origins, not complete Unicode conformance or a complete historical Monaco build.
@@ -0,0 +1,46 @@
UNICODE, INC. LICENSE AGREEMENT - DATA FILES AND SOFTWARE
See Terms of Use for definitions of Unicode Inc.'s
Data Files and Software.
NOTICE TO USER: Carefully read the following legal agreement.
BY DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING UNICODE INC.'S
DATA FILES ("DATA FILES"), AND/OR SOFTWARE ("SOFTWARE"),
YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
TERMS AND CONDITIONS OF THIS AGREEMENT.
IF YOU DO NOT AGREE, DO NOT DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE
THE DATA FILES OR SOFTWARE.
COPYRIGHT AND PERMISSION NOTICE
Copyright © 1991-2020 Unicode, Inc. All rights reserved.
Distributed under the Terms of Use in https://www.unicode.org/copyright.html.
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Unicode data files and any associated documentation
(the "Data Files") or Unicode software and any associated documentation
(the "Software") to deal in the Data Files or Software
without restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, and/or sell copies of
the Data Files or Software, and to permit persons to whom the Data Files
or Software are furnished to do so, provided that either
(a) this copyright and permission notice appear with all copies
of the Data Files or Software, or
(b) this copyright and permission notice appear in associated
Documentation.
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT OF THIRD PARTY RIGHTS.
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS
NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL
DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
PERFORMANCE OF THE DATA FILES OR SOFTWARE.
Except as contained in this notice, the name of a copyright holder
shall not be used in advertising or otherwise to promote the sale,
use or other dealings in these Data Files or Software without prior
written authorization of the copyright holder.
@@ -0,0 +1,630 @@
var fs = require('fs');
var regexpu = require('regexpu');
// Generate a map of all codes that appear in emojis
var emojiCodesMap = {};
(function () {
var emojiTestData = fs.readFileSync('./emoji-test.txt').toString('utf8');
var emojiTestEntries = emojiTestData.split('\n');
for (var i = 0, len = emojiTestEntries.length; i < len; i++) {
var entry = emojiTestEntries[i];
// strip comments
entry = entry.replace(/#.*/, '');
entry = entry.trim();
// strip status
entry = entry.replace(/;.*/, '');
entry = entry.trim();
if (entry.indexOf(' ') > 0) {
// ignore combined
continue;
}
if (entry.length === 0) {
// ignore empty lines
continue;
}
emojiCodesMap[parseInt(entry, 16)] = true;
}
})();
function toUnicodeCode(n) {
return n.toString(16).toUpperCase();
}
// Read entire UnicodeData.txt and find contiguous ranges of emojis
var emojiCodes = [];
var BREAK_SIGNAL = -1;
(function () {
var data = fs.readFileSync('./UnicodeData.txt').toString('utf8');
var entries = data.split('\n');
var prevWasEmoji = false;
for (var i = 0, len = entries.length; i < len; i++) {
var entry = entries[i];
var props = entry.split(';');
var code = parseInt(props[0], 16);
var _isEmoji = isEmoji(code, emojiCodesMap[code]);
if (_isEmoji) {
emojiCodes.push(code);
prevWasEmoji = true;
} else if (prevWasEmoji) {
prevWasEmoji = false;
emojiCodes.push(BREAK_SIGNAL);
}
}
fs.writeFileSync('emoji-codes.txt', emojiCodes.filter(code => code !== BREAK_SIGNAL).map(toUnicodeCode).join('\n'));
})();
var emojiCodeRanges = [];
(function () {
var startCode = emojiCodes[0];
var prevCode = emojiCodes[0];
for (var i = 1; i < emojiCodes.length; i++) {
var code = emojiCodes[i];
if (code === BREAK_SIGNAL) {
continue;
}
if (prevCode + 1 !== code) {
emojiCodeRanges.push({
from: startCode,
to: prevCode
});
startCode = code;
}
prevCode = code;
}
emojiCodeRanges.push({
from: startCode,
to: prevCode
});
})();
fs.writeFileSync('emoji-code-ranges.txt', emojiCodeRanges.map(function (entry) {
var from = toUnicodeCode(entry.from);
var to = toUnicodeCode(entry.to);
if (from === to) {
return from;
}
return from + '-' + to;
}).join('\n'));
var fuzzy = [], fuzzyLength = 0, breakSignal = false;
fuzzy[fuzzyLength++] = {
from: emojiCodes[0],
to: emojiCodes[0]
};
for (var i = 1; i < emojiCodes.length; i++) {
var code = emojiCodes[i];
if (code === BREAK_SIGNAL) {
breakSignal = true;
continue;
}
if (breakSignal) {
fuzzy[fuzzyLength++] = {
from: code,
to: code
};
} else {
fuzzy[fuzzyLength - 1].to = code;
}
breakSignal = false;
}
fs.writeFileSync('emoji-code-ranges-fuzzy.txt', fuzzy.map(function (entry) {
var from = toUnicodeCode(entry.from);
var to = toUnicodeCode(entry.to);
if (from === to) {
return from;
}
return from + '-' + to;
}).join('\n'));
var isEmojiFuzzy = fuzzy.map(function(entry) {
var from = '\\u{'+toUnicodeCode(entry.from)+'}';
var to = '\\u{'+toUnicodeCode(entry.to)+'}';
if (from === to) {
return from;
}
return from + '-' + to;
}).join('');
var isEmojiVeryImprecise = fuzzy.map(function(entry) {
if (entry.from + 1 === entry.to) {
return `(x === ${entry.from}) || (x === ${entry.to})`;
}
if (entry.from === entry.to) {
return `(x === ${entry.from})`;
}
return `(x >= ${entry.from} && x <= ${entry.to})`;
}).join(' || ');
console.log('very imprecise test :: (x >= 0x1F1E6 && x <= 0x1F1FF) || ' + isEmojiVeryImprecise);
// Flags use two code points, but they usually start with 1F1E6 - 1F1FF
// REGIONAL INDICATOR SYMBOL LETTER A -> REGIONAL INDICATOR SYMBOL LETTER Z
isEmojiFuzzy += '\\u{1F1E6}-\\u{1F1FF}';
// TODO:
// use U+FE0F as a hint
// http://emojipedia.org/variation-selector-16/
// Variation Selector-16
// An invisible codepoint which specifies that the preceding character should be displayed with emoji presentation. Only required if the preceding character defaults to text presentation.
var regex = '[' + isEmojiFuzzy + ']';
console.log('UNICODE CODE-POINTS RANGES:');
console.log(regex);
var r = regexpu.rewritePattern(regex, 'u');
console.log('------');
console.log(r);
console.log('------');
// GOTTEN FROM ABOVE RESULT
var containsEmoji = /(?:[\u231A\u231B\u23F0\u23F3\u2600-\u27BF\u2B50\u2B55]|\uD83C[\uDDE6-\uDDFF\uDF00-\uDFFF]|\uD83D[\uDC00-\uDE4F\uDE80-\uDEFC\uDFE0-\uDFEB]|\uD83E[\uDD00-\uDDFF\uDE70-\uDED6])/;
// SANITY CHECK the regex
(function () {
var emojiTestData = fs.readFileSync('./emoji-test.txt').toString('utf8');
var emojiTestEntries = emojiTestData.split('\n');
var MISSING = [];
for (var i = 0, len = emojiTestEntries.length; i < len; i++) {
var original = emojiTestEntries[i];
var entry = emojiTestEntries[i];
// strip comments
entry = entry.replace(/#.*/, '');
entry = entry.trim();
// strip status
entry = entry.replace(/;.*/, '');
entry = entry.trim();
if (entry.length === 0) {
// ignore empty lines
continue;
}
var codes = entry.split(' ').map(function(code) {
return parseInt(code, 16);
});
var str = 'some ';
for (var j = 0; j < codes.length; j++) {
str += String.fromCodePoint(codes[j]);
}
str += ' text';
if (containsEmoji.test(str)) {
// regex captures this input
continue;
}
MISSING.push(original);
// MISSING.push('TEST STRING: <<' + str + '>>');
// console.log(str);
// String.fromCodePoint()
// console.log(entry);
// if (entry.indexOf(' ') > 0) {
// // ignore combined
// continue;
// }
// if (entry.length === 0) {
// // ignore empty lines
// continue;
// }
// emojiCodesMap[parseInt(entry, 16)] = true;
}
fs.writeFileSync('emoji-missed.txt', 'THE FOLLOWING ARE NOT CAPTURED BY THE REGEX: \n' + MISSING.join('\n'));
// console.log(MISSING);
})();
function isEmoji(code, isInEmojiTest) {
var block = getCodeBlock(code);
// Exclude some codes in emoji-test.txt that don't look like emojis.
if (block === 'Latin-1 Supplement') {
// e.g. 00A9 © copyright
return false;
}
if (block === 'General Punctuation') {
// e.g. 203C ‼️ double exclamation mark
return false;
}
if (block === 'Letterlike Symbols') {
// e.g. 2122 ️️™️ trade mark
return false;
}
if (block === 'Arrows') {
// e.g. 2194 ↔ left-right arrow
return false;
}
if (block === 'Miscellaneous Technical') {
// U+2300 -> U+23FE
if (code === 0x231A) { // ⌚ watch
return true;
}
if (code === 0x231B) { // ⌛ hourglass
return true;
}
if (code === 0x23F0) { // ⏰ alarm clock
return true;
}
if (code === 0x23F3) { // ⏳ hourglass with flowing sand
return true;
}
return false;
}
if (block === 'Enclosed Alphanumerics') {
// e.g. 24C2 Ⓜ circled M
return false;
}
if (block === 'Geometric Shapes') {
// e.g. 25AA ▪ black small square
return false;
}
if (block === 'Supplemental Arrows-B') {
// e.g. 2934 ⤴ right arrow curving up
return false;
}
if (block === 'Miscellaneous Symbols and Arrows') {
if (code === 0x2B50) { // ⭐ white medium star
return true;
}
if (code === 0x2B55) { // ⭕ heavy large circle
return true;
}
return false;
}
if (block === 'CJK Symbols and Punctuation') {
// e.g. 303D 〽 part alternation mark
return false;
}
if (block === 'Enclosed CJK Letters and Months') {
// e.g. 3297 ㊗ Japanese “congratulations” button
return false;
}
if (block === 'Mahjong Tiles') {
// e.g. 1F004 🀄 mahjong red dragon
return false;
}
if (block === 'Playing Cards') {
// e.g. 1F0CF 🃏 joker
return false;
}
if (block === 'Enclosed Alphanumeric Supplement') {
// e.g. 1F170 🅰 A button (blood type)
return false;
}
if (block === 'Enclosed Ideographic Supplement') {
// e.g. 1F201 🈁 Japanese “here” button
return false;
}
// include some blocks that look like emojis
if (block === 'Miscellaneous Symbols') {
return true;
}
if (block === 'Dingbats') {
return true;
}
if (block === 'Miscellaneous Symbols and Pictographs') {
return true;
}
if (block === 'Emoticons') {
return true;
}
if (block === 'Transport and Map Symbols') {
return true;
}
if (block === 'Supplemental Symbols and Pictographs') {
return true;
}
return isInEmojiTest;
}
// http://www.fileformat.info/info/unicode/block/index.htm
// replace ^([^\t]+)\tU\+([0-9A-Z]+)\tU\+([0-9A-Z]+)
// with if (code >= 0x$2 && code <= 0x$3) return '$1';
function getCodeBlock(code) {
if (code >= 0x0000 && code <= 0x007F) return 'Basic Latin';
if (code >= 0x0080 && code <= 0x00FF) return 'Latin-1 Supplement';
if (code >= 0x0100 && code <= 0x017F) return 'Latin Extended-A';
if (code >= 0x0180 && code <= 0x024F) return 'Latin Extended-B';
if (code >= 0x0250 && code <= 0x02AF) return 'IPA Extensions';
if (code >= 0x02B0 && code <= 0x02FF) return 'Spacing Modifier Letters';
if (code >= 0x0300 && code <= 0x036F) return 'Combining Diacritical Marks';
if (code >= 0x0370 && code <= 0x03FF) return 'Greek and Coptic';
if (code >= 0x0400 && code <= 0x04FF) return 'Cyrillic';
if (code >= 0x0500 && code <= 0x052F) return 'Cyrillic Supplement';
if (code >= 0x0530 && code <= 0x058F) return 'Armenian';
if (code >= 0x0590 && code <= 0x05FF) return 'Hebrew';
if (code >= 0x0600 && code <= 0x06FF) return 'Arabic';
if (code >= 0x0700 && code <= 0x074F) return 'Syriac';
if (code >= 0x0750 && code <= 0x077F) return 'Arabic Supplement';
if (code >= 0x0780 && code <= 0x07BF) return 'Thaana';
if (code >= 0x07C0 && code <= 0x07FF) return 'NKo';
if (code >= 0x0800 && code <= 0x083F) return 'Samaritan';
if (code >= 0x0840 && code <= 0x085F) return 'Mandaic';
if (code >= 0x08A0 && code <= 0x08FF) return 'Arabic Extended-A';
if (code >= 0x0900 && code <= 0x097F) return 'Devanagari';
if (code >= 0x0980 && code <= 0x09FF) return 'Bengali';
if (code >= 0x0A00 && code <= 0x0A7F) return 'Gurmukhi';
if (code >= 0x0A80 && code <= 0x0AFF) return 'Gujarati';
if (code >= 0x0B00 && code <= 0x0B7F) return 'Oriya';
if (code >= 0x0B80 && code <= 0x0BFF) return 'Tamil';
if (code >= 0x0C00 && code <= 0x0C7F) return 'Telugu';
if (code >= 0x0C80 && code <= 0x0CFF) return 'Kannada';
if (code >= 0x0D00 && code <= 0x0D7F) return 'Malayalam';
if (code >= 0x0D80 && code <= 0x0DFF) return 'Sinhala';
if (code >= 0x0E00 && code <= 0x0E7F) return 'Thai';
if (code >= 0x0E80 && code <= 0x0EFF) return 'Lao';
if (code >= 0x0F00 && code <= 0x0FFF) return 'Tibetan';
if (code >= 0x1000 && code <= 0x109F) return 'Myanmar';
if (code >= 0x10A0 && code <= 0x10FF) return 'Georgian';
if (code >= 0x1100 && code <= 0x11FF) return 'Hangul Jamo';
if (code >= 0x1200 && code <= 0x137F) return 'Ethiopic';
if (code >= 0x1380 && code <= 0x139F) return 'Ethiopic Supplement';
if (code >= 0x13A0 && code <= 0x13FF) return 'Cherokee';
if (code >= 0x1400 && code <= 0x167F) return 'Unified Canadian Aboriginal Syllabics';
if (code >= 0x1680 && code <= 0x169F) return 'Ogham';
if (code >= 0x16A0 && code <= 0x16FF) return 'Runic';
if (code >= 0x1700 && code <= 0x171F) return 'Tagalog';
if (code >= 0x1720 && code <= 0x173F) return 'Hanunoo';
if (code >= 0x1740 && code <= 0x175F) return 'Buhid';
if (code >= 0x1760 && code <= 0x177F) return 'Tagbanwa';
if (code >= 0x1780 && code <= 0x17FF) return 'Khmer';
if (code >= 0x1800 && code <= 0x18AF) return 'Mongolian';
if (code >= 0x18B0 && code <= 0x18FF) return 'Unified Canadian Aboriginal Syllabics Extended';
if (code >= 0x1900 && code <= 0x194F) return 'Limbu';
if (code >= 0x1950 && code <= 0x197F) return 'Tai Le';
if (code >= 0x1980 && code <= 0x19DF) return 'New Tai Lue';
if (code >= 0x19E0 && code <= 0x19FF) return 'Khmer Symbols';
if (code >= 0x1A00 && code <= 0x1A1F) return 'Buginese';
if (code >= 0x1A20 && code <= 0x1AAF) return 'Tai Tham';
if (code >= 0x1AB0 && code <= 0x1AFF) return 'Combining Diacritical Marks Extended';
if (code >= 0x1B00 && code <= 0x1B7F) return 'Balinese';
if (code >= 0x1B80 && code <= 0x1BBF) return 'Sundanese';
if (code >= 0x1BC0 && code <= 0x1BFF) return 'Batak';
if (code >= 0x1C00 && code <= 0x1C4F) return 'Lepcha';
if (code >= 0x1C50 && code <= 0x1C7F) return 'Ol Chiki';
if (code >= 0x1C80 && code <= 0x1C8F) return 'Cyrillic Extended-C';
if (code >= 0x1CC0 && code <= 0x1CCF) return 'Sundanese Supplement';
if (code >= 0x1CD0 && code <= 0x1CFF) return 'Vedic Extensions';
if (code >= 0x1D00 && code <= 0x1D7F) return 'Phonetic Extensions';
if (code >= 0x1D80 && code <= 0x1DBF) return 'Phonetic Extensions Supplement';
if (code >= 0x1DC0 && code <= 0x1DFF) return 'Combining Diacritical Marks Supplement';
if (code >= 0x1E00 && code <= 0x1EFF) return 'Latin Extended Additional';
if (code >= 0x1F00 && code <= 0x1FFF) return 'Greek Extended';
if (code >= 0x2000 && code <= 0x206F) return 'General Punctuation';
if (code >= 0x2070 && code <= 0x209F) return 'Superscripts and Subscripts';
if (code >= 0x20A0 && code <= 0x20CF) return 'Currency Symbols';
if (code >= 0x20D0 && code <= 0x20FF) return 'Combining Diacritical Marks for Symbols';
if (code >= 0x2100 && code <= 0x214F) return 'Letterlike Symbols';
if (code >= 0x2150 && code <= 0x218F) return 'Number Forms';
if (code >= 0x2190 && code <= 0x21FF) return 'Arrows';
if (code >= 0x2200 && code <= 0x22FF) return 'Mathematical Operators';
if (code >= 0x2300 && code <= 0x23FF) return 'Miscellaneous Technical';
if (code >= 0x2400 && code <= 0x243F) return 'Control Pictures';
if (code >= 0x2440 && code <= 0x245F) return 'Optical Character Recognition';
if (code >= 0x2460 && code <= 0x24FF) return 'Enclosed Alphanumerics';
if (code >= 0x2500 && code <= 0x257F) return 'Box Drawing';
if (code >= 0x2580 && code <= 0x259F) return 'Block Elements';
if (code >= 0x25A0 && code <= 0x25FF) return 'Geometric Shapes';
if (code >= 0x2600 && code <= 0x26FF) return 'Miscellaneous Symbols';
if (code >= 0x2700 && code <= 0x27BF) return 'Dingbats';
if (code >= 0x27C0 && code <= 0x27EF) return 'Miscellaneous Mathematical Symbols-A';
if (code >= 0x27F0 && code <= 0x27FF) return 'Supplemental Arrows-A';
if (code >= 0x2800 && code <= 0x28FF) return 'Braille Patterns';
if (code >= 0x2900 && code <= 0x297F) return 'Supplemental Arrows-B';
if (code >= 0x2980 && code <= 0x29FF) return 'Miscellaneous Mathematical Symbols-B';
if (code >= 0x2A00 && code <= 0x2AFF) return 'Supplemental Mathematical Operators';
if (code >= 0x2B00 && code <= 0x2BFF) return 'Miscellaneous Symbols and Arrows';
if (code >= 0x2C00 && code <= 0x2C5F) return 'Glagolitic';
if (code >= 0x2C60 && code <= 0x2C7F) return 'Latin Extended-C';
if (code >= 0x2C80 && code <= 0x2CFF) return 'Coptic';
if (code >= 0x2D00 && code <= 0x2D2F) return 'Georgian Supplement';
if (code >= 0x2D30 && code <= 0x2D7F) return 'Tifinagh';
if (code >= 0x2D80 && code <= 0x2DDF) return 'Ethiopic Extended';
if (code >= 0x2DE0 && code <= 0x2DFF) return 'Cyrillic Extended-A';
if (code >= 0x2E00 && code <= 0x2E7F) return 'Supplemental Punctuation';
if (code >= 0x2E80 && code <= 0x2EFF) return 'CJK Radicals Supplement';
if (code >= 0x2F00 && code <= 0x2FDF) return 'Kangxi Radicals';
if (code >= 0x2FF0 && code <= 0x2FFF) return 'Ideographic Description Characters';
if (code >= 0x3000 && code <= 0x303F) return 'CJK Symbols and Punctuation';
if (code >= 0x3040 && code <= 0x309F) return 'Hiragana';
if (code >= 0x30A0 && code <= 0x30FF) return 'Katakana';
if (code >= 0x3100 && code <= 0x312F) return 'Bopomofo';
if (code >= 0x3130 && code <= 0x318F) return 'Hangul Compatibility Jamo';
if (code >= 0x3190 && code <= 0x319F) return 'Kanbun';
if (code >= 0x31A0 && code <= 0x31BF) return 'Bopomofo Extended';
if (code >= 0x31C0 && code <= 0x31EF) return 'CJK Strokes';
if (code >= 0x31F0 && code <= 0x31FF) return 'Katakana Phonetic Extensions';
if (code >= 0x3200 && code <= 0x32FF) return 'Enclosed CJK Letters and Months';
if (code >= 0x3300 && code <= 0x33FF) return 'CJK Compatibility';
if (code >= 0x3400 && code <= 0x4DBF) return 'CJK Unified Ideographs Extension A';
if (code >= 0x4DC0 && code <= 0x4DFF) return 'Yijing Hexagram Symbols';
if (code >= 0x4E00 && code <= 0x9FFF) return 'CJK Unified Ideographs';
if (code >= 0xA000 && code <= 0xA48F) return 'Yi Syllables';
if (code >= 0xA490 && code <= 0xA4CF) return 'Yi Radicals';
if (code >= 0xA4D0 && code <= 0xA4FF) return 'Lisu';
if (code >= 0xA500 && code <= 0xA63F) return 'Vai';
if (code >= 0xA640 && code <= 0xA69F) return 'Cyrillic Extended-B';
if (code >= 0xA6A0 && code <= 0xA6FF) return 'Bamum';
if (code >= 0xA700 && code <= 0xA71F) return 'Modifier Tone Letters';
if (code >= 0xA720 && code <= 0xA7FF) return 'Latin Extended-D';
if (code >= 0xA800 && code <= 0xA82F) return 'Syloti Nagri';
if (code >= 0xA830 && code <= 0xA83F) return 'Common Indic Number Forms';
if (code >= 0xA840 && code <= 0xA87F) return 'Phags-pa';
if (code >= 0xA880 && code <= 0xA8DF) return 'Saurashtra';
if (code >= 0xA8E0 && code <= 0xA8FF) return 'Devanagari Extended';
if (code >= 0xA900 && code <= 0xA92F) return 'Kayah Li';
if (code >= 0xA930 && code <= 0xA95F) return 'Rejang';
if (code >= 0xA960 && code <= 0xA97F) return 'Hangul Jamo Extended-A';
if (code >= 0xA980 && code <= 0xA9DF) return 'Javanese';
if (code >= 0xA9E0 && code <= 0xA9FF) return 'Myanmar Extended-B';
if (code >= 0xAA00 && code <= 0xAA5F) return 'Cham';
if (code >= 0xAA60 && code <= 0xAA7F) return 'Myanmar Extended-A';
if (code >= 0xAA80 && code <= 0xAADF) return 'Tai Viet';
if (code >= 0xAAE0 && code <= 0xAAFF) return 'Meetei Mayek Extensions';
if (code >= 0xAB00 && code <= 0xAB2F) return 'Ethiopic Extended-A';
if (code >= 0xAB30 && code <= 0xAB6F) return 'Latin Extended-E';
if (code >= 0xAB70 && code <= 0xABBF) return 'Cherokee Supplement';
if (code >= 0xABC0 && code <= 0xABFF) return 'Meetei Mayek';
if (code >= 0xAC00 && code <= 0xD7AF) return 'Hangul Syllables';
if (code >= 0xD7B0 && code <= 0xD7FF) return 'Hangul Jamo Extended-B';
if (code >= 0xD800 && code <= 0xDB7F) return 'High Surrogates';
if (code >= 0xDB80 && code <= 0xDBFF) return 'High Private Use Surrogates';
if (code >= 0xDC00 && code <= 0xDFFF) return 'Low Surrogates';
if (code >= 0xE000 && code <= 0xF8FF) return 'Private Use Area';
if (code >= 0xF900 && code <= 0xFAFF) return 'CJK Compatibility Ideographs';
if (code >= 0xFB00 && code <= 0xFB4F) return 'Alphabetic Presentation Forms';
if (code >= 0xFB50 && code <= 0xFDFF) return 'Arabic Presentation Forms-A';
if (code >= 0xFE00 && code <= 0xFE0F) return 'Variation Selectors';
if (code >= 0xFE10 && code <= 0xFE1F) return 'Vertical Forms';
if (code >= 0xFE20 && code <= 0xFE2F) return 'Combining Half Marks';
if (code >= 0xFE30 && code <= 0xFE4F) return 'CJK Compatibility Forms';
if (code >= 0xFE50 && code <= 0xFE6F) return 'Small Form Variants';
if (code >= 0xFE70 && code <= 0xFEFF) return 'Arabic Presentation Forms-B';
if (code >= 0xFF00 && code <= 0xFFEF) return 'Halfwidth and Fullwidth Forms';
if (code >= 0xFFF0 && code <= 0xFFFF) return 'Specials';
if (code >= 0x10000 && code <= 0x1007F) return 'Linear B Syllabary';
if (code >= 0x10080 && code <= 0x100FF) return 'Linear B Ideograms';
if (code >= 0x10100 && code <= 0x1013F) return 'Aegean Numbers';
if (code >= 0x10140 && code <= 0x1018F) return 'Ancient Greek Numbers';
if (code >= 0x10190 && code <= 0x101CF) return 'Ancient Symbols';
if (code >= 0x101D0 && code <= 0x101FF) return 'Phaistos Disc';
if (code >= 0x10280 && code <= 0x1029F) return 'Lycian';
if (code >= 0x102A0 && code <= 0x102DF) return 'Carian';
if (code >= 0x102E0 && code <= 0x102FF) return 'Coptic Epact Numbers';
if (code >= 0x10300 && code <= 0x1032F) return 'Old Italic';
if (code >= 0x10330 && code <= 0x1034F) return 'Gothic';
if (code >= 0x10350 && code <= 0x1037F) return 'Old Permic';
if (code >= 0x10380 && code <= 0x1039F) return 'Ugaritic';
if (code >= 0x103A0 && code <= 0x103DF) return 'Old Persian';
if (code >= 0x10400 && code <= 0x1044F) return 'Deseret';
if (code >= 0x10450 && code <= 0x1047F) return 'Shavian';
if (code >= 0x10480 && code <= 0x104AF) return 'Osmanya';
if (code >= 0x104B0 && code <= 0x104FF) return 'Osage';
if (code >= 0x10500 && code <= 0x1052F) return 'Elbasan';
if (code >= 0x10530 && code <= 0x1056F) return 'Caucasian Albanian';
if (code >= 0x10600 && code <= 0x1077F) return 'Linear A';
if (code >= 0x10800 && code <= 0x1083F) return 'Cypriot Syllabary';
if (code >= 0x10840 && code <= 0x1085F) return 'Imperial Aramaic';
if (code >= 0x10860 && code <= 0x1087F) return 'Palmyrene';
if (code >= 0x10880 && code <= 0x108AF) return 'Nabataean';
if (code >= 0x108E0 && code <= 0x108FF) return 'Hatran';
if (code >= 0x10900 && code <= 0x1091F) return 'Phoenician';
if (code >= 0x10920 && code <= 0x1093F) return 'Lydian';
if (code >= 0x10980 && code <= 0x1099F) return 'Meroitic Hieroglyphs';
if (code >= 0x109A0 && code <= 0x109FF) return 'Meroitic Cursive';
if (code >= 0x10A00 && code <= 0x10A5F) return 'Kharoshthi';
if (code >= 0x10A60 && code <= 0x10A7F) return 'Old South Arabian';
if (code >= 0x10A80 && code <= 0x10A9F) return 'Old North Arabian';
if (code >= 0x10AC0 && code <= 0x10AFF) return 'Manichaean';
if (code >= 0x10B00 && code <= 0x10B3F) return 'Avestan';
if (code >= 0x10B40 && code <= 0x10B5F) return 'Inscriptional Parthian';
if (code >= 0x10B60 && code <= 0x10B7F) return 'Inscriptional Pahlavi';
if (code >= 0x10B80 && code <= 0x10BAF) return 'Psalter Pahlavi';
if (code >= 0x10C00 && code <= 0x10C4F) return 'Old Turkic';
if (code >= 0x10C80 && code <= 0x10CFF) return 'Old Hungarian';
if (code >= 0x10E60 && code <= 0x10E7F) return 'Rumi Numeral Symbols';
if (code >= 0x11000 && code <= 0x1107F) return 'Brahmi';
if (code >= 0x11080 && code <= 0x110CF) return 'Kaithi';
if (code >= 0x110D0 && code <= 0x110FF) return 'Sora Sompeng';
if (code >= 0x11100 && code <= 0x1114F) return 'Chakma';
if (code >= 0x11150 && code <= 0x1117F) return 'Mahajani';
if (code >= 0x11180 && code <= 0x111DF) return 'Sharada';
if (code >= 0x111E0 && code <= 0x111FF) return 'Sinhala Archaic Numbers';
if (code >= 0x11200 && code <= 0x1124F) return 'Khojki';
if (code >= 0x11280 && code <= 0x112AF) return 'Multani';
if (code >= 0x112B0 && code <= 0x112FF) return 'Khudawadi';
if (code >= 0x11300 && code <= 0x1137F) return 'Grantha';
if (code >= 0x11400 && code <= 0x1147F) return 'Newa';
if (code >= 0x11480 && code <= 0x114DF) return 'Tirhuta';
if (code >= 0x11580 && code <= 0x115FF) return 'Siddham';
if (code >= 0x11600 && code <= 0x1165F) return 'Modi';
if (code >= 0x11660 && code <= 0x1167F) return 'Mongolian Supplement';
if (code >= 0x11680 && code <= 0x116CF) return 'Takri';
if (code >= 0x11700 && code <= 0x1173F) return 'Ahom';
if (code >= 0x118A0 && code <= 0x118FF) return 'Warang Citi';
if (code >= 0x11AC0 && code <= 0x11AFF) return 'Pau Cin Hau';
if (code >= 0x11C00 && code <= 0x11C6F) return 'Bhaiksuki';
if (code >= 0x11C70 && code <= 0x11CBF) return 'Marchen';
if (code >= 0x12000 && code <= 0x123FF) return 'Cuneiform';
if (code >= 0x12400 && code <= 0x1247F) return 'Cuneiform Numbers and Punctuation';
if (code >= 0x12480 && code <= 0x1254F) return 'Early Dynastic Cuneiform';
if (code >= 0x13000 && code <= 0x1342F) return 'Egyptian Hieroglyphs';
if (code >= 0x14400 && code <= 0x1467F) return 'Anatolian Hieroglyphs';
if (code >= 0x16800 && code <= 0x16A3F) return 'Bamum Supplement';
if (code >= 0x16A40 && code <= 0x16A6F) return 'Mro';
if (code >= 0x16AD0 && code <= 0x16AFF) return 'Bassa Vah';
if (code >= 0x16B00 && code <= 0x16B8F) return 'Pahawh Hmong';
if (code >= 0x16F00 && code <= 0x16F9F) return 'Miao';
if (code >= 0x16FE0 && code <= 0x16FFF) return 'Ideographic Symbols and Punctuation';
if (code >= 0x17000 && code <= 0x187FF) return 'Tangut';
if (code >= 0x18800 && code <= 0x18AFF) return 'Tangut Components';
if (code >= 0x1B000 && code <= 0x1B0FF) return 'Kana Supplement';
if (code >= 0x1BC00 && code <= 0x1BC9F) return 'Duployan';
if (code >= 0x1BCA0 && code <= 0x1BCAF) return 'Shorthand Format Controls';
if (code >= 0x1D000 && code <= 0x1D0FF) return 'Byzantine Musical Symbols';
if (code >= 0x1D100 && code <= 0x1D1FF) return 'Musical Symbols';
if (code >= 0x1D200 && code <= 0x1D24F) return 'Ancient Greek Musical Notation';
if (code >= 0x1D300 && code <= 0x1D35F) return 'Tai Xuan Jing Symbols';
if (code >= 0x1D360 && code <= 0x1D37F) return 'Counting Rod Numerals';
if (code >= 0x1D400 && code <= 0x1D7FF) return 'Mathematical Alphanumeric Symbols';
if (code >= 0x1D800 && code <= 0x1DAAF) return 'Sutton SignWriting';
if (code >= 0x1E000 && code <= 0x1E02F) return 'Glagolitic Supplement';
if (code >= 0x1E800 && code <= 0x1E8DF) return 'Mende Kikakui';
if (code >= 0x1E900 && code <= 0x1E95F) return 'Adlam';
if (code >= 0x1EE00 && code <= 0x1EEFF) return 'Arabic Mathematical Alphabetic Symbols';
if (code >= 0x1F000 && code <= 0x1F02F) return 'Mahjong Tiles';
if (code >= 0x1F030 && code <= 0x1F09F) return 'Domino Tiles';
if (code >= 0x1F0A0 && code <= 0x1F0FF) return 'Playing Cards';
if (code >= 0x1F100 && code <= 0x1F1FF) return 'Enclosed Alphanumeric Supplement';
if (code >= 0x1F200 && code <= 0x1F2FF) return 'Enclosed Ideographic Supplement';
if (code >= 0x1F300 && code <= 0x1F5FF) return 'Miscellaneous Symbols and Pictographs';
if (code >= 0x1F600 && code <= 0x1F64F) return 'Emoticons';
if (code >= 0x1F650 && code <= 0x1F67F) return 'Ornamental Dingbats';
if (code >= 0x1F680 && code <= 0x1F6FF) return 'Transport and Map Symbols';
if (code >= 0x1F700 && code <= 0x1F77F) return 'Alchemical Symbols';
if (code >= 0x1F780 && code <= 0x1F7FF) return 'Geometric Shapes Extended';
if (code >= 0x1F800 && code <= 0x1F8FF) return 'Supplemental Arrows-C';
if (code >= 0x1F900 && code <= 0x1F9FF) return 'Supplemental Symbols and Pictographs';
if (code >= 0x20000 && code <= 0x2A6DF) return 'CJK Unified Ideographs Extension B';
if (code >= 0x2A700 && code <= 0x2B73F) return 'CJK Unified Ideographs Extension C';
if (code >= 0x2B740 && code <= 0x2B81F) return 'CJK Unified Ideographs Extension D';
if (code >= 0x2B820 && code <= 0x2CEAF) return 'CJK Unified Ideographs Extension E';
if (code >= 0x2F800 && code <= 0x2FA1F) return 'CJK Compatibility Ideographs Supplement';
if (code >= 0xE0000 && code <= 0xE007F) return 'Tags';
if (code >= 0xE0100 && code <= 0xE01EF) return 'Variation Selectors Supplement';
if (code >= 0xF0000 && code <= 0xFFFFF) return 'Supplementary Private Use Area-A';
if (code >= 0x100000 && code <= 0x10FFFF) return 'Supplementary Private Use Area-B';
}
@@ -0,0 +1,142 @@
const fs = require('fs');
const assert = require('assert');
function parseLine(line) {
line = line.replace(/#.*/, '');
line = line.trim();
if (line.length === 0) {
return null;
}
let parts = line.split(';').map(s => s.trim());
if (/\.\./.test(parts[0])) {
const range = parts[0].split('..');
return { from: parseInt(range[0], 16), to: parseInt(range[1], 16), type: parts[1] };
}
return { from: parseInt(parts[0], 16), to: parseInt(parts[0], 16), type: parts[1] };
}
function parseFile(filename) {
const fileContents = fs.readFileSync(filename).toString('utf8');
const lines = fileContents.split('\n');
const result = [];
for (let i = 0; i < lines.length; i++) {
let line = parseLine(lines[i]);
if (line) {
result.push(line);
}
}
return result;
}
const typeToInt = {
'Prepend': 1,
'CR': 2,
'LF': 3,
'Control': 4,
'Extend': 5,
'Regional_Indicator': 6,
'SpacingMark': 7,
'L': 8,
'V': 9,
'T': 10,
'LV': 11,
'LVT': 12,
'ZWJ': 13,
'Extended_Pictographic': 14
}
const tuples = [];
parseFile('./GraphemeBreakProperty.txt').forEach(item => tuples.push({ from:item.from, to:item.to, type: typeToInt[item.type] }));
parseFile('./emoji-data.txt').filter(item => item.type === 'Extended_Pictographic').forEach(item => tuples.push({ from:item.from, to:item.to, type: typeToInt[item.type] }));
tuples.sort((t1, t2) => t1.from - t2.from);
function pow2smallereq(n) {
let pow = 0;
let r = 1;
while (r <= n) {
r = r << 1;
pow++;
}
return pow;
}
let tuplesTree = [];
function buildTree(fromIndex, toIndex, outIndex) {
const elementCount = toIndex - fromIndex + 1;
let midIndex;
if (elementCount === 1) {
midIndex = fromIndex;
} else if (elementCount === 2) {
midIndex = fromIndex + 1;
} else {
const subtreeDepth = pow2smallereq(elementCount) - 1;
const minRightCount = (1 << (subtreeDepth - 1)) - 1;
const idealLeftCount = (1 << subtreeDepth) - 1;
const leftCount = Math.min(idealLeftCount, elementCount - 1 - minRightCount);
// console.log(`elementCount: ${elementCount}, subtreeDepth: ${subtreeDepth}, idealLeftCount: ${idealLeftCount}, minRightCount: ${minRightCount} ==> leftCount: ${leftCount}`);
midIndex = fromIndex + leftCount;
}
tuplesTree[outIndex] = tuples[midIndex];
// if (outIndex !== 1) {
// const parentIndex = (outIndex / 2) | 0;
// if (outIndex === 2 * parentIndex) {
// // I am left node
// if (tuplesTree[outIndex].to >= tuplesTree[parentIndex].from) {
// console.log(`illegal tree 1!`);
// }
// } else {
// // I am right node
// if (tuplesTree[outIndex].from <= tuplesTree[parentIndex].to) {
// console.log(`illegal tree 2!`);
// }
// }
// }
if (fromIndex < midIndex) {
// left subtree
buildTree(fromIndex, midIndex - 1, 2 * outIndex);
}
if (midIndex < toIndex) {
// right subtree
buildTree(midIndex + 1, toIndex, 2 * outIndex + 1);
}
}
buildTree(0, tuples.length - 1, 1);
// for (let i = 1; i < tuplesTree.length; i++) {
// if (tuplesTree[i]) {
// console.log(`${i} - ${tuplesTree[i].from},${tuplesTree[i].to},${tuplesTree[i].type}`);
// } else {
// console.log(`${i} - undefined`);
// }
// }
function validateNode(nodeIndex) {
const node = tuplesTree[nodeIndex];
const leftChildIndex = 2 * nodeIndex;
if (leftChildIndex < tuplesTree.length) {
const leftChild = tuplesTree[leftChildIndex];
assert.ok(leftChild.to < node.from);
validateNode(leftChildIndex);
}
const rightChildIndex = 2 * nodeIndex + 1;
if (rightChildIndex < tuplesTree.length) {
const rightChild = tuplesTree[rightChildIndex];
assert.ok(rightChild.from > node.to);
validateNode(rightChildIndex);
}
}
validateNode(1);
let result = [0,0,0];
tuplesTree.forEach(t => result.push(t.from, t.to, t.type));
console.log(JSON.stringify(result));
@@ -0,0 +1,123 @@
var fs = require('fs');
var regexpu = require('regexpu');
var data = fs.readFileSync('./UnicodeData.txt').toString('utf8');
var entries = data.split('\n');
var rtlCodes = [], rtlCodesLength = 0;
var BREAK_SIGNAL = -1;
function toUnicodeCode(n) {
return n.toString(16).toUpperCase();
}
var prevWasRTL = false;
for (var i = 0, len = entries.length; i < len; i++) {
var entry = entries[i];
var props = entry.split(';');
var code = props[0];
var dir = props[4];
if (dir === 'R' || dir === 'AL') {
rtlCodes[rtlCodesLength++] = parseInt(code, 16);
prevWasRTL = true;
} else if (prevWasRTL) {
prevWasRTL = false;
rtlCodes[rtlCodesLength++] = BREAK_SIGNAL;
}
}
fs.writeFileSync('rtl-codes.txt', rtlCodes.filter(code => code !== BREAK_SIGNAL).map(toUnicodeCode).join('\n'));
var rtlCodeRanges = [], rtlCodeRangesLength = 0;
var startCode = rtlCodes[0];
var prevCode = rtlCodes[0];
for (var i = 1; i < rtlCodesLength; i++) {
var code = rtlCodes[i];
if (code === BREAK_SIGNAL) {
continue;
}
if (prevCode + 1 !== code) {
rtlCodeRanges[rtlCodeRangesLength++] = {
from: startCode,
to: prevCode
};
startCode = code;
}
prevCode = code;
}
rtlCodeRanges[rtlCodeRangesLength++] = {
from: startCode,
to: prevCode
};
fs.writeFileSync('rtl-code-ranges.txt', rtlCodeRanges.map(function(entry) {
var from = toUnicodeCode(entry.from);
var to = toUnicodeCode(entry.to);
if (from === to) {
return from;
}
return from + '-' + to;
}).join('\n'));
var fuzzy = [], fuzzyLength = 0, breakSignal = false;
fuzzy[fuzzyLength++] = {
from: rtlCodes[0],
to: rtlCodes[0]
};
for (var i = 1; i < rtlCodesLength; i++) {
var code = rtlCodes[i];
if (code === BREAK_SIGNAL) {
breakSignal = true;
continue;
}
if (breakSignal) {
fuzzy[fuzzyLength++] = {
from: code,
to: code
};
} else {
fuzzy[fuzzyLength - 1].to = code;
}
breakSignal = false;
}
fs.writeFileSync('rtl-code-ranges-fuzzy.txt', fuzzy.map(function(entry) {
var from = toUnicodeCode(entry.from);
var to = toUnicodeCode(entry.to);
if (from === to) {
return from;
}
return from + '-' + to;
}).join('\n'));
var isRTLFuzzy = fuzzy.map(function(entry) {
var from = '\\u{'+toUnicodeCode(entry.from)+'}';
var to = '\\u{'+toUnicodeCode(entry.to)+'}';
if (from === to) {
return from;
}
return from + '-' + to;
}).join('');
var regex = '[' + isRTLFuzzy + ']';
console.log('UNICODE CODE-POINTS RANGES:');
console.log(regex);
var r = regexpu.rewritePattern(regex, 'u');
console.log('------');
console.log(r);
console.log('------');
// var containsRTL = /(?:[\u05BE\u05C0\u05C3\u05C6\u05D0-\u05F4\u0608\u060B\u060D\u061B-\u064A\u066D-\u066F\u0671-\u06D5\u06E5\u06E6\u06EE\u06EF\u06FA-\u0710\u0712-\u072F\u074D-\u07A5\u07B1-\u07EA\u07F4\u07F5\u07FA-\u0815\u081A\u0824\u0828\u0830-\u0858\u085E-\u08BD\u200F\uFB1D\uFB1F-\uFB28\uFB2A-\uFD3D\uFD50-\uFDFC\uFE70-\uFEFC]|\uD802[\uDC00-\uDD1B\uDD20-\uDE00\uDE10-\uDE33\uDE40-\uDEE4\uDEEB-\uDF35\uDF40-\uDFFF]|\uD803[\uDC00-\uDCFF]|\uD83A[\uDC00-\uDCCF\uDD00-\uDD43\uDD50-\uDFFF]|\uD83B[\uDC00-\uDEBB])/
// var r = containsRTL.test('a');
// console.log(r);
//console.log(rtlCodeRanges);
@@ -0,0 +1,124 @@
{
"name": "unicode-utils",
"version": "1.0.0",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
"ast-types": {
"version": "0.9.6",
"resolved": "https://registry.npmjs.org/ast-types/-/ast-types-0.9.6.tgz",
"integrity": "sha1-ECyenpAF0+fjgpvwxPok7oYu6bk="
},
"esprima": {
"version": "3.1.3",
"resolved": "https://registry.npmjs.org/esprima/-/esprima-3.1.3.tgz",
"integrity": "sha1-/cpRzuYTOJXjyI1TXOSdv/YqRjM="
},
"jsesc": {
"version": "0.5.0",
"resolved": "https://registry.npmjs.org/jsesc/-/jsesc-0.5.0.tgz",
"integrity": "sha1-597mbjXW/Bb3EP6R1c9p9w8IkR0="
},
"private": {
"version": "0.1.8",
"resolved": "https://registry.npmjs.org/private/-/private-0.1.8.tgz",
"integrity": "sha512-VvivMrbvd2nKkiG38qjULzlc+4Vx4wm/whI9pQD35YrARNnhxeiRktSOhSukRLFNlzg6Br/cJPet5J/u19r/mg=="
},
"recast": {
"version": "0.11.23",
"resolved": "https://registry.npmjs.org/recast/-/recast-0.11.23.tgz",
"integrity": "sha1-RR/TAEqx5N+bTktmN2sqIZEkYtM=",
"requires": {
"ast-types": "0.9.6",
"esprima": "~3.1.0",
"private": "~0.1.5",
"source-map": "~0.5.0"
}
},
"regenerate": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/regenerate/-/regenerate-1.4.0.tgz",
"integrity": "sha512-1G6jJVDWrt0rK99kBjvEtziZNCICAuvIPkSiUFIQxVP06RCVpq3dmDo2oi6ABpYaDYaTRr67BEhL8r1wgEZZKg=="
},
"regenerate-unicode-properties": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/regenerate-unicode-properties/-/regenerate-unicode-properties-2.0.0.tgz",
"integrity": "sha1-MFmsn7NIM69QpqAJgM9FwxmpF4Y=",
"requires": {
"regenerate": "^1.3.1"
}
},
"regexpu": {
"version": "3.3.0",
"resolved": "https://registry.npmjs.org/regexpu/-/regexpu-3.3.0.tgz",
"integrity": "sha1-ib2Wh8E5o6BnpaF2+Jm7L4hnOAc=",
"requires": {
"recast": "^0.11.12",
"regexpu-core": "^3.3.0"
}
},
"regexpu-core": {
"version": "3.3.0",
"resolved": "https://registry.npmjs.org/regexpu-core/-/regexpu-core-3.3.0.tgz",
"integrity": "sha1-7fAAIU4osRe38JLIQAzNE9x091Q=",
"requires": {
"regenerate": "^1.3.1",
"regenerate-unicode-properties": "^2.0.0",
"regjsgen": "^0.3.0",
"regjsparser": "^0.2.0",
"unicode-match-property": "^0.1.1",
"unicode-match-property-value": "^1.0.1"
}
},
"regjsgen": {
"version": "0.3.0",
"resolved": "https://registry.npmjs.org/regjsgen/-/regjsgen-0.3.0.tgz",
"integrity": "sha1-DuSj6SdkMM2iXx54nqbBW4ewy0M="
},
"regjsparser": {
"version": "0.2.1",
"resolved": "https://registry.npmjs.org/regjsparser/-/regjsparser-0.2.1.tgz",
"integrity": "sha1-w3h1U/rwTndcMCEC7zRtmVAA7Bw=",
"requires": {
"jsesc": "~0.5.0"
}
},
"source-map": {
"version": "0.5.7",
"resolved": "https://registry.npmjs.org/source-map/-/source-map-0.5.7.tgz",
"integrity": "sha1-igOdLRAh0i0eoUyA2OpGi6LvP8w="
},
"unicode-canonical-property-names": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/unicode-canonical-property-names/-/unicode-canonical-property-names-1.0.3.tgz",
"integrity": "sha1-Y7DwKXHEgSRT3yIevT75oWs8ouw="
},
"unicode-match-property": {
"version": "0.1.3",
"resolved": "https://registry.npmjs.org/unicode-match-property/-/unicode-match-property-0.1.3.tgz",
"integrity": "sha1-NxxzCFIrZaLMTPu753UYf3eENSo=",
"requires": {
"unicode-canonical-property-names": "^1.0.3",
"unicode-property-aliases": "^1.1.0"
}
},
"unicode-match-property-value": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/unicode-match-property-value/-/unicode-match-property-value-1.0.2.tgz",
"integrity": "sha1-Vd4wCwQRKoy2Y8AQI0PAgnN3FMw=",
"requires": {
"unicode-property-value-aliases": "^1.2.1"
}
},
"unicode-property-aliases": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/unicode-property-aliases/-/unicode-property-aliases-1.4.0.tgz",
"integrity": "sha512-Wwwqwp89bM1GWmm/Lle7KSJKkUS0SRIK+KVHp0ryxy6k9Daa1fh/n8vt46wS5aswK7gzIWHWQBtRdXexgXdf3A=="
},
"unicode-property-value-aliases": {
"version": "1.2.2",
"resolved": "https://registry.npmjs.org/unicode-property-value-aliases/-/unicode-property-value-aliases-1.2.2.tgz",
"integrity": "sha1-EqN0b8ZvdfCKP0qNVngPifSay70="
}
}
}
@@ -0,0 +1,25 @@
{
"name": "unicode-utils",
"version": "1.0.0",
"description": "",
"main": "fetch-data.js",
"dependencies": {
"regexpu": "^3.3.0"
},
"devDependencies": {
"regexpu": "^4.5.4"
},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
"type": "git",
"url": "git+https://github.com/alexandrudima/unicode-utils.git"
},
"author": "Alex Dima",
"license": "MIT",
"bugs": {
"url": "https://github.com/alexandrudima/unicode-utils/issues"
},
"homepage": "https://github.com/alexandrudima/unicode-utils#readme"
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,67 @@
# SITE-07 Monaco 历史 Unicode 数据与通知
来源 HEAD:`1fed96a9093aaac474e36b5cb4013dc50eef60c0`。本次为 SITE-07 的
独立检查点,任务继续 doing、VENDOR-06 继续 open;没有更改编辑器、worker、
播放器运行字节、公开 API 或根锁文件。
## 已实现
冻结并复验 VS Code `829382514cb1065f5ebb90f436e1c6103e153953` 的 strings.ts、
原 Monaco 0.30.1 两份 source map 及实际交付的 editor.main/workerMain 文件。
三份原生成脚本复现了 RTL 正则、Emoji 正则、imprecise 谓词和全部 5,034 个断字表
整数。正则及整数精确比较,谓词通过 TS token 序列忽略排版和注释,不执行表达式。
| 数据 | 冻结来源 | 版本边界 |
| --- | --- | --- |
| RTL | unicode-utils `5c256860b1e4e54b0375165b8451cf7cdabc953a` | fetch 配方指定 UnicodeData-10.0.0d5,不能写成最终 Unicode 10 |
| Emoji 与 imprecise | `6d37cf4362203bfb5431c560cd8809b2cc3b51c1` | UnicodeData 13.0 与 Emoji 13.1 测试数据共同输入 |
| Grapheme tree | `2068fd0695b2798d8debb3646c860f5d37ba75b2` | 2019 年 GraphemeBreakProperty-13.0.0d4 和同批 emoji-data 快照 |
输入来自原生成项目冻结的 Git 数据;版本依据为对应 fetch 配方和数据文件头,
没有声称直接从今天的 unicode.org 重下了相同历史草案。记录同时保留完整历史
Unicode 数据/软件条款,来源为官方 unicodetools 固定 2021 提交,明确其参考身份。
生成器 package.json 写明 Alex Dima/MIT,但固定目录没有独立 LICENSE;不编造
原生成器的许可证文件或原始版权年份。原 Monaco/Microsoft 通知保留。
## 模块和依赖
- `unicode.ts`:AST 边界、token/regex/tree 比对及原配方输入输出适配。
- `unicode-origins.ts`:来源记录与两份运行资产、完整条款/说明的强制关联。
- `reproduce-unicode.ts`:15 个 Git blob、两个 Monaco 归档、17 个历史工具归档
的验证、隔离解包及三份原配方执行。新增 `yarn verify:monaco-unicode [--fetch]`。
- `monaco-unicode-provenance.json`:源文件 SHA-256/Git blob、归档 SHA-512/SHA-256、
原 lock integrity、生成结果及字节不变的交付文件记录。
- 正常通知构建加入 core-unicode 完整条款和说明;合计 90 份 notice 加索引。
历史工具使用原 lock 的 17 包闭包,其中 regexpu 实际解析为 3.3.0;package.json
的 dependencies/devDependencies 范围互相矛盾,不按当前范围重新解析。独立缓存
位于 `.cache/monaco-review/unicode-generator`,不运行安装/生命周期脚本,不改变
根 dependency/yarn.lock,不交付到站点运行环境。读取仅限已验证输入,写入仅捕获
在内存;拒绝未知 import、输入、重复/缺失输出。VM 不是恶意代码安全沙箱。
## 验证
Node 24.21.0、Yarn 1.22.22、现有 TS 5.9.3;34/34 单测通过,包含错误/缺失
声明、格式变化、数据替换、配方越界和遗漏通知的负例。严格 docs-tools 类型、
只读 lint、工具链、联网及离线复验、通知生成/只读检查通过。
Chromium 153.0.8010.12、Firefox 155.0、WebKit 26.6(Windows)共 6/6 通过,
零跳过/重试。加载实际 Monaco 模块检查 RTL、Emoji、断字分类与边界;真实编辑器
创建混合文字模型、插入中文并撤销;移动页实际播放、全部通知的 HTTP 字节保持
一致。具体观察及网络诊断见[机器证据](../baselines/monaco-unicode-validation.json)。
这是所选历史行为的验证,不是完整 Unicode conformance 测试。
实现途中发现并修正了两个复验器问题:原 lock 同时使用 SHA-1/SHA-512,不能
假设全是 SHA-1;Emoji 配方也读取 UnicodeData,不能只登记 emoji-test。另将
谓词比较从保留原换行的 printer 结果改为完整 token 序列,增加换行/注释负例。
这些是工具实现修正,没有修改原数据或放宽生成结果比较。
## 剩余与回退
本批关闭选定 Unicode 数据来源缺口;其他内嵌来源、原 WinJS 改编版本、完整
Monaco core 编译及 SITE-07 其他范围仍未完成,202/266 完成数不变。此检查点
不替代生态包验收、远端 CI、真机证据和三轮发布复盘。
回退时撤销本批验证器、冻结输入、通知绑定与测试,并重新生成 notices;原编辑器
资产始终未改。维护命令与边界见 [Monaco README](../../scripts/site-vendor/monaco/README.md)。
未推送、部署或发布。
+1 -1
View File
@@ -655,7 +655,7 @@
- SITE-AI-DOCS-01: [记录](changes/2026-09-14-SITE-AI-DOCS-01-documentation-pipeline.md) [记录](baselines/documentation-pipeline-validation.json)
- SITE-BUILD-01: [记录](changes/2026-09-14-SITE-BUILD-01-staged-builds.md) [记录](baselines/site-build-validation.json)
- SITE-03: [记录](changes/2026-09-14-SITE-03-desktop-editor.md) [记录](baselines/site-editor-validation.json)
- SITE-07: [记录](site-inventory.md) [记录](baselines/site-provenance.json) [记录](changes/2026-09-14-SITE-07-vendor-notices.md) [记录](baselines/site-notices-checkpoint.json) [记录](baselines/site-codicons-provenance.json) [记录](baselines/site-codicons-validation.json) [记录](changes/2026-09-15-SITE-07-codicons.md) [记录](baselines/vconsole-notices-provenance.json) [记录](baselines/vconsole-notices-validation.json) [记录](changes/2026-09-15-SITE-07-vconsole-notices.md) [记录](console-modernization.md) [记录](baselines/site-console-inventory.json) [记录](baselines/site-console-validation.json) [记录](changes/2026-09-15-SITE-07-console-baseline.md) [记录](baselines/console-feed-provenance.json) [记录](changes/2026-09-15-SITE-07-console-feed-source.md) [记录](baselines/console-commonjs-provenance.json) [记录](changes/2026-09-15-SITE-07-console-commonjs.md) [记录](baselines/console-esm-provenance.json) [记录](changes/2026-09-15-SITE-07-console-esm.md) [记录](baselines/console-embedded-notices.json) [记录](baselines/console-notices-validation.json) [记录](changes/2026-09-15-SITE-07-console-notices.md) [记录](baselines/console-embedded-sources.json) [记录](baselines/console-embedded-validation.json) [记录](changes/2026-09-15-SITE-07-console-embedded-sources.md) [记录](baselines/console-derived-attribution.json) [记录](baselines/console-derived-validation.json) [记录](changes/2026-09-15-SITE-07-console-derived-attribution.md) [记录](baselines/console-stackoverflow-provenance.json) [记录](baselines/console-stackoverflow-validation.json) [记录](changes/2026-09-15-SITE-07-console-stackoverflow.md) [记录](baselines/console-shallowequal-validation.json) [记录](changes/2026-09-15-SITE-07-console-shallowequal.md) [记录](console-notice-review.md) [记录](baselines/console-notice-review-validation.json) [记录](changes/2026-09-15-SITE-07-console-notice-review.md) [记录](baselines/monaco-typescript-provenance.json) [记录](baselines/monaco-typescript-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-typescript.md) [记录](baselines/monaco-languages-provenance.json) [记录](baselines/monaco-languages-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-languages.md) [记录](baselines/monaco-language-notices.json) [记录](baselines/monaco-language-notices-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-language-notices.md) [记录](baselines/monaco-modes-provenance.json) [记录](baselines/monaco-modes-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-modes.md) [记录](baselines/monaco-basic-provenance.json) [记录](baselines/monaco-basic-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-basic.md) [记录](baselines/monaco-core-origins-provenance.json) [记录](baselines/monaco-core-origins-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-core-origins.md) [记录](baselines/monaco-core-build-provenance.json) [记录](baselines/monaco-core-build-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-core-build.md) [记录](baselines/monaco-contributions-provenance.json) [记录](baselines/monaco-contributions-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-contributions.md) [记录](baselines/monaco-css-build-provenance.json) [记录](baselines/monaco-css-build-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-css.md) [记录](baselines/monaco-node-path-provenance.json) [记录](baselines/monaco-node-path-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-node-path.md) [记录](changes/2026-09-15-SITE-07-monaco-dom.md) [记录](baselines/monaco-dom-origins-validation.json)
- SITE-07: [记录](site-inventory.md) [记录](baselines/site-provenance.json) [记录](changes/2026-09-14-SITE-07-vendor-notices.md) [记录](baselines/site-notices-checkpoint.json) [记录](baselines/site-codicons-provenance.json) [记录](baselines/site-codicons-validation.json) [记录](changes/2026-09-15-SITE-07-codicons.md) [记录](baselines/vconsole-notices-provenance.json) [记录](baselines/vconsole-notices-validation.json) [记录](changes/2026-09-15-SITE-07-vconsole-notices.md) [记录](console-modernization.md) [记录](baselines/site-console-inventory.json) [记录](baselines/site-console-validation.json) [记录](changes/2026-09-15-SITE-07-console-baseline.md) [记录](baselines/console-feed-provenance.json) [记录](changes/2026-09-15-SITE-07-console-feed-source.md) [记录](baselines/console-commonjs-provenance.json) [记录](changes/2026-09-15-SITE-07-console-commonjs.md) [记录](baselines/console-esm-provenance.json) [记录](changes/2026-09-15-SITE-07-console-esm.md) [记录](baselines/console-embedded-notices.json) [记录](baselines/console-notices-validation.json) [记录](changes/2026-09-15-SITE-07-console-notices.md) [记录](baselines/console-embedded-sources.json) [记录](baselines/console-embedded-validation.json) [记录](changes/2026-09-15-SITE-07-console-embedded-sources.md) [记录](baselines/console-derived-attribution.json) [记录](baselines/console-derived-validation.json) [记录](changes/2026-09-15-SITE-07-console-derived-attribution.md) [记录](baselines/console-stackoverflow-provenance.json) [记录](baselines/console-stackoverflow-validation.json) [记录](changes/2026-09-15-SITE-07-console-stackoverflow.md) [记录](baselines/console-shallowequal-validation.json) [记录](changes/2026-09-15-SITE-07-console-shallowequal.md) [记录](console-notice-review.md) [记录](baselines/console-notice-review-validation.json) [记录](changes/2026-09-15-SITE-07-console-notice-review.md) [记录](baselines/monaco-typescript-provenance.json) [记录](baselines/monaco-typescript-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-typescript.md) [记录](baselines/monaco-languages-provenance.json) [记录](baselines/monaco-languages-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-languages.md) [记录](baselines/monaco-language-notices.json) [记录](baselines/monaco-language-notices-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-language-notices.md) [记录](baselines/monaco-modes-provenance.json) [记录](baselines/monaco-modes-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-modes.md) [记录](baselines/monaco-basic-provenance.json) [记录](baselines/monaco-basic-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-basic.md) [记录](baselines/monaco-core-origins-provenance.json) [记录](baselines/monaco-core-origins-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-core-origins.md) [记录](baselines/monaco-core-build-provenance.json) [记录](baselines/monaco-core-build-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-core-build.md) [记录](baselines/monaco-contributions-provenance.json) [记录](baselines/monaco-contributions-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-contributions.md) [记录](baselines/monaco-css-build-provenance.json) [记录](baselines/monaco-css-build-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-css.md) [记录](baselines/monaco-node-path-provenance.json) [记录](baselines/monaco-node-path-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-node-path.md) [记录](changes/2026-09-15-SITE-07-monaco-dom.md) [记录](baselines/monaco-dom-origins-validation.json) [记录](changes/2026-09-15-SITE-07-monaco-unicode.md) [记录](baselines/monaco-unicode-provenance.json) [记录](baselines/monaco-unicode-validation.json)
- EX-01: [记录](changes/2026-09-14-EX-01-react-consumer.md) [记录](baselines/react-consumer-validation.json) [记录](scripts/react-consumer.mjs)
- EX-02: [记录](changes/2026-09-14-EX-02-vue-consumer.md) [记录](baselines/vue-consumer-validation.json) [记录](scripts/vue-consumer.mjs)
- MOD-01: [记录](changes/2026-09-15-MOD-01-bun-evaluation.md) [记录](baselines/bun-install-validation.json) [记录](bun-evaluation.md)
+13
View File
@@ -1,5 +1,18 @@
# 进度与证据
## SITE-07 Monaco 历史 Unicode 数据检查点
三份原生成配方在冻结依赖下复现 RTL/Emoji 正则、imprecise token 表达式和全部
5,034 个断字整数,与固定 VS Code 源码、两份原 map 和实际交付 editor/worker
绑定。历史输入包含 2017/2019 草案与 2021 数据,保留差异而非统一换最新版。
新增完整 Unicode 历史条款/来源说明,正常通知交付为 90 份加索引。
34/34 单测、类型/lint/工具链、联网/离线复验及三浏览器编辑/撤销/字符分类、移动
实播和 HTTP notices 6/6 通过。见[检查点](changes/2026-09-15-SITE-07-monaco-unicode.md)
和[证据](baselines/monaco-unicode-validation.json)。17 个旧工具依赖仅隔离复验,
根依赖/锁和运行资产未改。SITE-07/VENDOR-06 仍开放,202 done、21 doing、43 todo;
下一步继续剩余来源和插件兼容验收,不将这一局部检查算成整体发布完成。
## SITE-07 Monaco DOM 改编来源与通知绑定
六段 DOM 声明与固定 VS Code 源码及原始 source map 精确对应,当前 editor.main.js 与原 npm 归档相同。新增 WinJS 4.4.5 固定参考源码/完整 MIT 条款和明确的未知原始改编版本说明;普通构建交付 88 份 notice 加索引。30/30 单测、类型/lint、联网/离线复验和三浏览器尺寸/滚动位置/移动实播/通知 HTTP 6/6 通过。见[记录](changes/2026-09-15-SITE-07-monaco-dom.md)和[证据](baselines/monaco-dom-origins-validation.json)。
+5 -2
View File
@@ -664,11 +664,14 @@
"refactor/changes/2026-09-15-SITE-07-monaco-node-path.md",
"refactor/changes/2026-09-15-SITE-07-monaco-dom.md",
"refactor/baselines/monaco-dom-origins-provenance.json",
"refactor/baselines/monaco-dom-origins-validation.json"
"refactor/baselines/monaco-dom-origins-validation.json",
"refactor/changes/2026-09-15-SITE-07-monaco-unicode.md",
"refactor/baselines/monaco-unicode-provenance.json",
"refactor/baselines/monaco-unicode-validation.json"
],
"compatibleResolution": "Keep separate from owned TS migration. Verify source/version/diff and license notices before replacement; preserve API/CSS/worker URLs and run owning package tests.",
"closureCriteria": "固定上游版本/内容差异、完整组件许可与分发 notices,兼容测试通过;仅当前上游许可证名称不足以关闭。",
"workspaceState": "Monaco 0.30.1 remains byte-pinned, including its known CSS newline-only difference. Codicons 0.0.26, TypeScript 4.4.4 and language-service licenses are delivered. Three service workers, four modes and 76 basic-language bundles reproduce exactly; all 13 core JS minified outputs reproduce with locked esbuild 0.12.6 on Windows x64. Loader/header and css/nls source origins match fixed Git; two maps inventory 584 prepared-source entries and 549 different names. DOMPurify 2.3.1 and marked 3.0.2 have exact source adaptations and complete notice supplements; 85 site notices were verified by HTTP/mobile browser checks. Latest contribution check verifies six archives and 102 Git members, emits 88 TS registration modules and uses RequireJS 2.3.6/Terser 5.9.0 to reproduce the entire dev/min editor assembly, including core string-table entry, dependency injection, order, alias and map trailer. Latest units 23/23, strict type/lint and offline/network source checks pass; modes/tokenizer regressions pass 6/6 across three engines with 2523 token cases per engine. Earlier ten-locale editing/find 30/30 and diff-worker 3/3 evidence remains recorded separately. CSS now reproduces from 68 styles, three embedded images and the copied font, with 81 verified Git members. The archived transport/plugin recipes match every prepared ESM stylesheet; explicit normalization of 70 generated CRLF separators reproduces the entire development CSS. cssnano 4.1.11/PostCSS 7.0.35 reproduce complete minified CSS using isolated frozen Yarn installs (153 selectors/143 versions) and historical browser defaults. The site retains its five header CRLF differences. The exact upstream stage that normalized the 70 development separators is still unidentified. Latest units 25/25, strict type/lint and offline/network checks pass; new three-engine style/theme/font/resize regressions pass 3/3. Original core TS compilation and remaining embedded-origin review stay open; this is not release approval or an exhaustive sanitizer audit. The Node path origin is now fixed to v14.16.0/bd60e93357a118204ea238d94e7a9e4209d93062 and the verified VS Code port. Its complete existing notice text matches both sources; a source supplement corrects the older index link without rewriting it. Six removed exports reproduce both source-map inputs; pinned TS 4.5.0-dev.20211021 emits exact full editor/worker modules. Thirty deterministic original Node path results pass in each browser engine, together with all 86 delivered notices and actual playback: 6/6 browser, 28/28 unit, type/lint, offline/network checks. Chromium retains one observed media ERR_ABORTED despite successful playback. Other embedded origins/full core compilation remain open. DOM helpers now bind six exact VS Code/source-map declarations and fixed WinJS4.4.5 comparison source/complete MIT terms to the unchanged editor; original adaptation version remains unknown, not inferred from that reference. Other origins and full original core compilation remain open."
"workspaceState": "Monaco 0.30.1 remains byte-pinned, including its known CSS newline-only difference. Codicons 0.0.26, TypeScript 4.4.4 and language-service licenses are delivered. Three service workers, four modes and 76 basic-language bundles reproduce exactly; all 13 core JS minified outputs reproduce with locked esbuild 0.12.6 on Windows x64. Loader/header and css/nls source origins match fixed Git; two maps inventory 584 prepared-source entries and 549 different names. DOMPurify 2.3.1 and marked 3.0.2 have exact source adaptations and complete notice supplements; 85 site notices were verified by HTTP/mobile browser checks. Latest contribution check verifies six archives and 102 Git members, emits 88 TS registration modules and uses RequireJS 2.3.6/Terser 5.9.0 to reproduce the entire dev/min editor assembly, including core string-table entry, dependency injection, order, alias and map trailer. Latest units 23/23, strict type/lint and offline/network source checks pass; modes/tokenizer regressions pass 6/6 across three engines with 2523 token cases per engine. Earlier ten-locale editing/find 30/30 and diff-worker 3/3 evidence remains recorded separately. CSS now reproduces from 68 styles, three embedded images and the copied font, with 81 verified Git members. The archived transport/plugin recipes match every prepared ESM stylesheet; explicit normalization of 70 generated CRLF separators reproduces the entire development CSS. cssnano 4.1.11/PostCSS 7.0.35 reproduce complete minified CSS using isolated frozen Yarn installs (153 selectors/143 versions) and historical browser defaults. The site retains its five header CRLF differences. The exact upstream stage that normalized the 70 development separators is still unidentified. Latest units 25/25, strict type/lint and offline/network checks pass; new three-engine style/theme/font/resize regressions pass 3/3. Original core TS compilation and remaining embedded-origin review stay open; this is not release approval or an exhaustive sanitizer audit. The Node path origin is now fixed to v14.16.0/bd60e93357a118204ea238d94e7a9e4209d93062 and the verified VS Code port. Its complete existing notice text matches both sources; a source supplement corrects the older index link without rewriting it. Six removed exports reproduce both source-map inputs; pinned TS 4.5.0-dev.20211021 emits exact full editor/worker modules. Thirty deterministic original Node path results pass in each browser engine, together with all 86 delivered notices and actual playback: 6/6 browser, 28/28 unit, type/lint, offline/network checks. Chromium retains one observed media ERR_ABORTED despite successful playback. Other embedded origins/full core compilation remain open. DOM helpers now bind six exact VS Code/source-map declarations and fixed WinJS4.4.5 comparison source/complete MIT terms to the unchanged editor; original adaptation version remains unknown, not inferred from that reference. Other origins and full original core compilation remain open. Unicode checkpoint reproduces RTL/Emoji predicates and all 5034 grapheme integers using mixed historical snapshots, matching both original maps and unchanged shipped editor/worker; complete historical Unicode terms and explicit source/version limitations are delivered. Other origins and full original core compilation remain open."
},
{
"id": "VENDOR-07",
+4 -1
View File
@@ -4558,7 +4558,10 @@
"baselines/monaco-node-path-validation.json",
"changes/2026-09-15-SITE-07-monaco-node-path.md",
"changes/2026-09-15-SITE-07-monaco-dom.md",
"baselines/monaco-dom-origins-validation.json"
"baselines/monaco-dom-origins-validation.json",
"changes/2026-09-15-SITE-07-monaco-unicode.md",
"baselines/monaco-unicode-provenance.json",
"baselines/monaco-unicode-validation.json"
]
},
{
+4 -2
View File
@@ -512,11 +512,13 @@
"SITE-05"
],
"sourceStatus": "Monaco 0.30.1 verified against official npm archive: 98 byte-exact files, one CSS newline-only difference. The bundled font additionally matches @vscode/codicons 0.0.26 exactly; see site-codicons-provenance.json.",
"licenseStatus": "Full Monaco root notices and Codicons historical notices are delivered. The actual TypeScript 4.4.4 worker is exactly reproduced and has a complete original LICENSE/CopyrightNotice/ThirdPartyNoticeText supplement plus attribution; the original stale 2.7.2 label is preserved with explanation. Other bundled-component completeness remains under VENDOR-06. No permission for unrelated fonts/media is inferred. CSS/HTML/JSON worker source proof now binds seven service packages to nine original license/third-party notice files, two embedded formatter headers and attribution. JSON glob BSD conditions and actual formatter author years are retained; all 81 site notices pass HTTP byte checks. Mode/core/basic-language and remaining embedded-source completeness stay open. All four language mode bundles are now exactly reproduced and the two shared npm dependencies are bound to those mode assets as well as workers. Core/basic-language and remaining upstream-origin review stay open. DOM adaptation supplement now delivers fixed WinJS4.4.5 reference terms and explicit unknown-original-version attribution; six fixed VS Code declaration blocks match the original source map.88 notice files plus index are served; remaining provenance is not waived.",
"licenseStatus": "Full Monaco root notices and Codicons historical notices are delivered. The actual TypeScript 4.4.4 worker is exactly reproduced and has a complete original LICENSE/CopyrightNotice/ThirdPartyNoticeText supplement plus attribution; the original stale 2.7.2 label is preserved with explanation. Other bundled-component completeness remains under VENDOR-06. No permission for unrelated fonts/media is inferred. CSS/HTML/JSON worker source proof now binds seven service packages to nine original license/third-party notice files, two embedded formatter headers and attribution. JSON glob BSD conditions and actual formatter author years are retained; all 81 site notices pass HTTP byte checks. Mode/core/basic-language and remaining embedded-source completeness stay open. All four language mode bundles are now exactly reproduced and the two shared npm dependencies are bound to those mode assets as well as workers. Core/basic-language and remaining upstream-origin review stay open. DOM adaptation supplement now delivers fixed WinJS4.4.5 reference terms and explicit unknown-original-version attribution; six fixed VS Code declaration blocks match the original source map.88 notice files plus index are served; remaining provenance is not waived. Unicode data supplement now delivers fixed historical Unicode terms and mixed-draft/input attribution; three original recipes reproduce all four selected expressions. Current notice delivery is 90 files plus index; remaining provenance is not waived.",
"upstreamSources": [
"https://registry.npmjs.org/monaco-editor/-/monaco-editor-0.30.1.tgz",
"https://registry.npmjs.org/@vscode/codicons/-/codicons-0.0.26.tgz",
"https://github.com/winjs/winjs/tree/4329b1133b243d9ded3b5a1f98d096ee8e80e889"
"https://github.com/winjs/winjs/tree/4329b1133b243d9ded3b5a1f98d096ee8e80e889",
"https://github.com/alexdima/unicode-utils/tree/6d37cf4362203bfb5431c560cd8809b2cc3b51c1",
"https://github.com/unicode-org/unicodetools/blob/b1c21ef7e972ea0c783c19cf25494bde645fbf21/LICENSE"
],
"upstreamReviewedAt": "2026-09-15",
"updatePolicy": "Keep separate from owned TS migration. Verify source/version/diff and license notices before replacement; preserve API/CSS/worker URLs and run owning package tests.",
+3
View File
@@ -6,6 +6,7 @@ import { verifyMonacoCoreNotices } from './site-vendor/monaco/core-origins.ts'
import { verifyMonacoDomNotices } from './site-vendor/monaco/dom-origins.ts'
import { verifyMonacoPathNotices } from './site-vendor/monaco/node-path.ts'
import { verifyMonacoLanguageNotices } from './site-vendor/monaco/notices.ts'
import { verifyMonacoUnicodeNotices } from './site-vendor/monaco/unicode-origins.ts'
import { writeOrCheckNotices } from './site-vendor/notices.ts'
assert(process.argv.slice(2).every(arg => arg === '--check'), 'Use yarn build:site-notices [--check]')
@@ -15,6 +16,7 @@ assert.deepEqual(manifest.groups.map(group => group.name).sort(), ['console', 'm
assert.deepEqual(manifest.groups.filter(group => group.name !== 'console').flatMap(group => (group.components || []).map(component => component.name)).sort(), [
'@babel/runtime',
'@vscode/codicons',
'Unicode data (Monaco core)',
'WinJS-derived DOM helpers (Monaco core)',
'copy-text-to-clipboard',
'core-js',
@@ -94,6 +96,7 @@ for (const name of ['LICENSE', 'MIT-LICENSE', 'ATTRIBUTION.md'])
verifyConsoleNoticeSources(process.cwd(), manifest)
verifyMonacoCoreNotices(process.cwd(), manifest)
verifyMonacoDomNotices(process.cwd(), manifest)
verifyMonacoUnicodeNotices(process.cwd(), manifest)
verifyMonacoPathNotices(process.cwd(), manifest)
verifyMonacoLanguageNotices(process.cwd(), manifest)
const count = writeOrCheckNotices(process.cwd(), manifest, process.argv.includes('--check'))
+23
View File
@@ -1035,6 +1035,16 @@
"source": "refactor/baselines/site-vendor/monaco-dom/ATTRIBUTION.md.txt",
"target": "docs/licenses/monaco-editor/core-dom/ATTRIBUTION.md",
"sha256": "0858d19261742809cecd7174a0a06b2b0013a4b9560b78aede1edd3dd350da21"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/LICENSE.txt",
"target": "docs/licenses/monaco-editor/core-unicode/LICENSE.txt",
"sha256": "80b381b2f157c31e7f3222bdb932bda9fbd7b335e888beec56820fb2993363de"
},
{
"source": "refactor/baselines/site-vendor/monaco-unicode/ATTRIBUTION.md.txt",
"target": "docs/licenses/monaco-editor/core-unicode/ATTRIBUTION.md",
"sha256": "51d583153fd1ed701a2b8810b3b1994d337002f074255e75a7d06790a381a1a6"
}
],
"review": "The complete upstream Monaco LICENSE/ThirdPartyNotices are retained. The unmodified bundled Codicons font exactly matches @vscode/codicons 0.0.26; its historical README, CC BY 4.0 content license, MIT code license and added attribution are distributed below. Other site assets remain subject to their separate provenance reviews.",
@@ -1239,6 +1249,19 @@
"docs/licenses/monaco-editor/core-dom/LICENSE.txt",
"docs/licenses/monaco-editor/core-dom/ATTRIBUTION.md"
]
},
{
"name": "Unicode data (Monaco core)",
"version": "mixed historical inputs: 10.0.0d5, 13.0 drafts, Emoji 13.1",
"tarball": "https://github.com/alexdima/unicode-utils",
"assets": [
"docs/assets/js/vs/editor/editor.main.js",
"docs/assets/js/vs/base/worker/workerMain.js"
],
"notices": [
"docs/licenses/monaco-editor/core-unicode/LICENSE.txt",
"docs/licenses/monaco-editor/core-unicode/ATTRIBUTION.md"
]
}
]
},
+48 -1
View File
@@ -388,7 +388,7 @@ Do not replace this distinction with a guessed version during later maintenance.
The ordinary notice build binds the reference license and attribution to the
editor asset; it rejects omitted terms or a changed source/asset relationship.
The site now carries 88 notice files plus its index. Frozen source and delivered
That checkpoint added two notices, bringing the site to 88 files plus its index. Frozen source and delivered
terms preserve original bytes through Git attributes. No dependencies or editor
runtime bytes changed; the verifier uses the existing TypeScript parser.
@@ -406,3 +406,50 @@ native geometry. The site test checks all notice HTTP bytes, relative links and
real mobile-page playback. Unicode/other embedded sources, unknown original
WinJS version, complete original core compilation and SITE-07 acceptance remain
open. Source comments and this bounded proof are not full provenance clearance.
## Historical Unicode data
`unicode.ts` extracts the four top-level data expressions using TypeScript 5.9.3
AST boundaries. It compares regex text exactly, the imprecise emoji predicate as
tokens (ignoring formatting/comments only), and every integer of the grapheme
tree. `unicode-origins.ts` binds the complete historical Unicode terms and
attribution to the shipped editor and base worker. The ordinary notice build now
delivers 90 notice files plus the index; it does not run old dependencies.
`reproduce-unicode.ts` checks 15 fixed Git blobs, two npm archives, both original
source maps and both shipped assets. It replays three original generator scripts
against frozen inputs. Their dependency closure consists of 17 locked archives,
including regexpu 3.3.0. Original lock integrity (SHA-1 or SHA-512) plus independent
SHA-512/SHA-256 is verified before individual member extraction into the isolated
`refactor/.cache/monaco-review/unicode-generator/compiler/node_modules` tree.
No install/lifecycle scripts run, and root dependencies/yarn.lock are unchanged.
The input versions deliberately differ: RTL uses the 2017 Unicode 10.0.0d5
snapshot, emoji uses Unicode 13.0 plus Emoji 13.1, and grapheme data uses the 2019
Unicode 13 draft snapshot. That last combination reproduces all 5,034 integers.
Version labels come from the frozen fetch recipes and file headers; do not
replace them with a guessed single final Unicode release. Generator package
metadata declares MIT and Alex Dima; its frozen tree has no standalone LICENSE.
The complete Unicode terms are a fixed 2021 official repository reference,
explicitly identified as such rather than asserted to be an original bundled file.
Generator execution allows only the recorded data reads, captures expected output
files in memory and rejects unknown imports, unused inputs and missing/repeated
outputs. The VM is an I/O adapter for verified historical code, not a security
sandbox for arbitrary scripts. Never point it at unverified executable sources.
```sh
yarn verify:monaco-unicode --fetch
yarn verify:monaco-unicode
node --test test/monaco-unicode.test.js test/monaco-provenance.test.js test/site-notices.test.js
yarn typecheck:docs-tools
yarn check:site-notices
yarn test:browser editor-unicode.spec.js site-vendor.spec.js --workers=1
```
Offline reproduction needs the verified archive cache from the first command.
Unit tests guard data substitution, expression boundaries and recipe I/O. Browser
tests load the actual shipped Monaco module, exercise RTL/emoji/grapheme cases and
mixed-text model edit/undo, and verify HTTP notices plus mobile-page playback.
These checks are not full Unicode conformance or full original Monaco compilation;
other embedded origins and the remaining SITE-07 work stay open.
@@ -0,0 +1,58 @@
import assert from 'node:assert/strict'
import { createHash } from 'node:crypto'
import fs from 'node:fs'
import { createRequire } from 'node:module'
import path from 'node:path'
import process from 'node:process'
import { fileURLToPath } from 'node:url'
import { ArchiveCache } from './archives.ts'
import { readUnicodeOrigins, verifyMonacoUnicodeNotices } from './unicode-origins.ts'
import { readUnicodeTables, replayUnicodeGenerator, verifyUnicodeGeneration } from './unicode.ts'
const root = fileURLToPath(new URL('../../../', import.meta.url))
assert(process.argv.slice(2).every(arg => arg === '--fetch'), 'Use reproduce-unicode.ts [--fetch]')
assert.equal(process.version, `v${fs.readFileSync(path.join(root, '.node-version'), 'utf8').trim()}`, 'Use canonical Node')
const record = readUnicodeOrigins(root)
const cache = new ArchiveCache(root, path.join(root, 'refactor/.cache/monaco-review'))
const generators = new ArchiveCache(root, path.join(cache.directory, 'unicode-generator'))
await cache.verify(record.archives, record.remotes, process.argv.includes('--fetch'))
await generators.verify(record.generatorArchives, [], process.argv.includes('--fetch'))
verifyMonacoUnicodeNotices(root, JSON.parse(fs.readFileSync(path.join(root, 'scripts/site-vendor/manifest.json'), 'utf8')))
const lock: { lockfileVersion: number, dependencies: Record<string, { version: string, resolved: string, integrity: string, dependencies?: unknown }> } = JSON.parse(fs.readFileSync(path.join(root, record.lockSource), 'utf8'))
assert.equal(lock.lockfileVersion, 1)
assert.deepEqual(record.generatorArchives.map(archive => archive.name).sort(), Object.keys(lock.dependencies).sort(), 'Incomplete Unicode dependency closure')
for (const archive of record.generatorArchives) {
const dependency = lock.dependencies[archive.name]!
assert.equal(dependency.dependencies, undefined, 'Unexpected nested Unicode dependency')
assert.deepEqual([archive.version, archive.tarball, archive.lockedIntegrity], [dependency.version, dependency.resolved, dependency.integrity], 'Changed Unicode lock resolution')
const bytes = fs.readFileSync(path.join(generators.directory, `${archive.name}-${archive.version}.tgz`))
const algorithm = archive.lockedIntegrity.split('-')[0]!
assert(['sha1', 'sha512'].includes(algorithm), 'Unexpected original Unicode integrity algorithm')
assert.equal(`${algorithm}-${createHash(algorithm).update(bytes).digest('base64')}`, archive.lockedIntegrity, 'Changed original Unicode archive integrity')
generators.extractCompiler(archive)
}
const require = createRequire(path.join(generators.directory, 'compiler/entry.cjs'))
assert.equal(require('regexpu/package.json').version, '3.3.0')
const recipeSources = new Set(record.remotes.map(remote => remote.source))
assert.deepEqual(record.generators.map(generator => generator.kind), ['rtl', 'emoji', 'grapheme'])
const outputs = record.generators.map((generator) => {
assert(recipeSources.has(generator.source), 'Unverified Unicode generator')
const inputs = new Map(Object.entries(generator.inputs).map(([name, source]) => {
assert(recipeSources.has(source), 'Unverified Unicode data input')
return [name, fs.readFileSync(path.join(root, source))]
}))
return replayUnicodeGenerator(fs.readFileSync(path.join(root, generator.source), 'utf8'), inputs, generator.outputs, require('regexpu'))
})
const tables = readUnicodeTables(fs.readFileSync(path.join(root, record.vscodeSource), 'utf8'))
verifyUnicodeGeneration(tables, outputs[0]!, outputs[1]!, outputs[2]!)
assert.deepEqual(record.maps.map(entry => entry.member), ['package/dev/vs/editor/editor.main.js.map', 'package/dev/vs/base/worker/workerMain.js.map'], 'Incomplete Unicode map coverage')
for (const entry of record.maps) {
const map: { sources: string[], sourcesContent: string[] } = JSON.parse(cache.member(entry).toString('utf8'))
assert.equal(map.sources.filter(source => source === entry.source).length, 1, 'Missing or repeated Unicode mapped source')
assert.deepEqual(readUnicodeTables(map.sourcesContent[map.sources.indexOf(entry.source)]!), tables, 'Mapped Unicode differs from fixed VS Code')
}
assert.deepEqual(record.shipped.map(asset => asset.target), record.component.assets, 'Incomplete Unicode shipped coverage')
for (const asset of record.shipped)
assert.deepEqual(fs.readFileSync(path.join(root, asset.target)), cache.member(asset), 'Shipped Unicode asset changed')
console.log(JSON.stringify({ archives: record.archives.length, generatorArchives: record.generatorArchives.length, gitSources: record.remotes.length, maps: record.maps.length, shippedAssets: record.shipped.length, exactGenerated: record.generated, fullMonacoCompilation: false, unicodeConformance: false }))
@@ -0,0 +1,46 @@
import type { VendorManifest } from '../notices.ts'
import type { Archive, Member, Remote } from './archives.ts'
import type { UnicodeTables } from './unicode.ts'
import assert from 'node:assert/strict'
import { Buffer } from 'node:buffer'
import fs from 'node:fs'
import path from 'node:path'
import { hash } from './archives.ts'
import { readUnicodeTables } from './unicode.ts'
export interface UnicodeOrigins {
archives: Archive[]
generatorArchives: (Archive & { lockedIntegrity: string })[]
remotes: Remote[]
vscodeSource: string
packageSource: string
lockSource: string
generators: { kind: 'rtl' | 'emoji' | 'grapheme', source: string, inputs: Record<string, string>, outputs: string[] }[]
maps: (Member & { source: string })[]
shipped: (Member & { target: string })[]
generated: Record<keyof UnicodeTables, { sha256: string, length: number }>
component: { name: string, version: string, tarball: string, assets: string[], notices: string[] }
notices: { source: string, target: string, sha256: string }[]
}
export function readUnicodeOrigins(root: string): UnicodeOrigins {
return JSON.parse(fs.readFileSync(path.join(root, 'refactor/baselines/monaco-unicode-provenance.json'), 'utf8'))
}
export function verifyMonacoUnicodeNotices(root: string, manifest: VendorManifest): void {
const record = readUnicodeOrigins(root)
assert.equal(record.component.name, 'Unicode data (Monaco core)')
assert.equal(record.component.version, 'mixed historical inputs: 10.0.0d5, 13.0 drafts, Emoji 13.1')
assert.deepEqual(record.component.assets, ['docs/assets/js/vs/editor/editor.main.js', 'docs/assets/js/vs/base/worker/workerMain.js'], 'Wrong Unicode origin assets')
assert.deepEqual(record.notices.map(notice => notice.target), ['docs/licenses/monaco-editor/core-unicode/LICENSE.txt', 'docs/licenses/monaco-editor/core-unicode/ATTRIBUTION.md'], 'Missing complete Unicode notice set')
assert.deepEqual(record.component.notices, record.notices.map(notice => notice.target), 'Missing Unicode attribution or terms')
const group = manifest.groups.find(group => group.name === 'monaco-editor')
assert(group, 'Missing Monaco Unicode group')
assert.deepEqual(group.components?.filter(item => item.name === record.component.name), [record.component], 'Wrong Monaco Unicode notice binding')
for (const notice of record.notices) {
assert.deepEqual(group.notices.filter(item => item.target === notice.target), [notice], 'Missing Monaco Unicode notice')
assert.equal(hash(fs.readFileSync(path.join(root, notice.source))), notice.sha256, 'Changed Monaco Unicode terms')
}
const tables = readUnicodeTables(fs.readFileSync(path.join(root, record.vscodeSource), 'utf8'))
assert.deepEqual(record.generated, Object.fromEntries(Object.entries(tables).map(([name, value]) => [name, { sha256: hash(Buffer.from(typeof value === 'string' ? value : JSON.stringify(value))), length: value.length }])), 'Changed Unicode data fragments')
}
+102
View File
@@ -0,0 +1,102 @@
import assert from 'node:assert/strict'
import vm from 'node:vm'
import ts from 'typescript'
export interface UnicodeTables { rtl: string, emoji: string, imprecise: string, grapheme: number[] }
export interface UnicodeOutput { logs: string[], writes: Map<string, string> }
function expressionText(expression: ts.Expression, file: ts.SourceFile): string {
while (ts.isParenthesizedExpression(expression))
expression = expression.expression
const scanner = ts.createScanner(ts.ScriptTarget.Latest, true, ts.LanguageVariant.Standard, expression.getText(file))
const tokens: string[] = []
while (scanner.scan() !== ts.SyntaxKind.EndOfFileToken)
tokens.push(scanner.getTokenText())
return tokens.join(' ')
}
export function readUnicodeTables(source: string): UnicodeTables {
const file = ts.createSourceFile('strings.ts', source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TS)
const found = new Map<string, ts.Expression>()
const put = (name: string, value: ts.Expression | undefined) => {
assert(value && !found.has(name), 'Missing or repeated Unicode declaration')
found.set(name, value)
}
for (const node of file.statements) {
if (ts.isVariableStatement(node)) {
for (const declaration of node.declarationList.declarations) {
if (ts.isIdentifier(declaration.name) && ['CONTAINS_RTL', 'CONTAINS_EMOJI'].includes(declaration.name.text))
put(declaration.name.text, declaration.initializer)
}
}
if (ts.isFunctionDeclaration(node) && node.name && ['isEmojiImprecise', 'getGraphemeBreakRawData'].includes(node.name.text)) {
assert.equal(node.body?.statements.length, 1, 'Unexpected Unicode function body')
const statement = node.body.statements[0]!
assert(ts.isReturnStatement(statement), 'Expected Unicode return')
put(node.name.text, statement.expression)
}
}
assert.equal(found.size, 4, 'Missing Unicode declarations')
const regex = (name: string) => {
const node = found.get(name)!
assert(ts.isRegularExpressionLiteral(node), 'Expected Unicode regex literal')
assert(node.text.startsWith('/') && node.text.endsWith('/'), 'Unexpected Unicode regex flags')
return node.text.slice(1, -1)
}
const tree = found.get('getGraphemeBreakRawData')!
assert(ts.isCallExpression(tree) && tree.expression.getText(file) === 'JSON.parse' && tree.arguments.length === 1 && ts.isStringLiteral(tree.arguments[0]!), 'Expected literal Unicode tree')
const grapheme: unknown = JSON.parse(tree.arguments[0].text)
assert(Array.isArray(grapheme) && grapheme.length > 0 && grapheme.length % 3 === 0 && grapheme.every(value => Number.isSafeInteger(value) && value >= 0), 'Invalid Unicode tree integers')
return { rtl: regex('CONTAINS_RTL'), emoji: regex('CONTAINS_EMOJI'), imprecise: expressionText(found.get('isEmojiImprecise')!, file), grapheme }
}
// Execute only hash-verified upstream recipes. This VM controls recipe I/O, not hostile code.
export function replayUnicodeGenerator(source: string, inputs: Map<string, Uint8Array>, outputs: string[], regexpu: unknown): UnicodeOutput {
const logs: string[] = []
const writes = new Map<string, string>()
const reads = new Set<string>()
const io = {
readFileSync(name: string) {
assert(inputs.has(name), `Unexpected Unicode input: ${name}`)
reads.add(name)
return inputs.get(name)!
},
writeFileSync(name: string, value: string) {
assert(outputs.includes(name) && !writes.has(name), 'Unexpected or repeated Unicode output')
assert.equal(typeof value, 'string', 'Expected Unicode output text')
writes.set(name, value)
},
}
vm.runInNewContext(source, {
require(name: string) {
if (name === 'fs')
return io
if (name === 'assert')
return assert
assert.equal(name, 'regexpu', 'Unexpected Unicode import')
return regexpu
},
console: { log: (...args: unknown[]) => logs.push(args.map(String).join(' ')) },
}, { timeout: 15000 })
assert.deepEqual([...reads].sort(), [...inputs.keys()].sort(), 'Unused Unicode input')
assert.deepEqual([...writes.keys()].sort(), [...outputs].sort(), 'Missing Unicode output')
return { logs, writes }
}
export function verifyUnicodeGeneration(tables: UnicodeTables, rtl: UnicodeOutput, emoji: UnicodeOutput, grapheme: UnicodeOutput): void {
const pattern = (output: UnicodeOutput) => {
const markers = output.logs.flatMap((line, index) => line === '------' ? [index] : [])
assert(markers.length === 2 && markers[1] === markers[0]! + 2, 'Missing Unicode regex delimiters')
return output.logs[markers[0]! + 1]
}
assert.equal(pattern(rtl), tables.rtl, 'Generated RTL differs from Monaco')
assert.equal(pattern(emoji), tables.emoji, 'Generated emoji differs from Monaco')
const expressions = emoji.logs.filter(line => line.startsWith('very imprecise test :: '))
assert.equal(expressions.length, 1, 'Missing or repeated imprecise emoji expression')
const file = ts.createSourceFile('expression.ts', expressions[0]!.slice('very imprecise test :: '.length), ts.ScriptTarget.Latest, true, ts.ScriptKind.TS)
assert.equal(file.statements.length, 1, 'Unexpected imprecise emoji expression')
const statement = file.statements[0]!
assert(ts.isExpressionStatement(statement), 'Expected imprecise emoji expression')
assert.equal(expressionText(statement.expression, file), tables.imprecise, 'Generated imprecise emoji differs from Monaco')
assert.deepEqual(JSON.parse(grapheme.logs[grapheme.logs.length - 1]!), tables.grapheme, 'Generated grapheme tree differs from Monaco')
}
+50
View File
@@ -0,0 +1,50 @@
import { expect, test } from './fixtures.js'
test('docs Monaco retains historical RTL, emoji and grapheme data during model edits', async ({ page }, testInfo) => {
await page.goto('/test/player.html?core=candidate&chapter=published')
await page.addScriptTag({ url: '/assets/js/vs/loader.js' })
const observed = await page.evaluate(async () => {
window.require.config({ paths: { vs: '/assets/js/vs' } })
const load = ids => new Promise((resolve, reject) => window.require(ids, (...modules) => resolve(modules), reject))
await load(['vs/editor/editor.main'])
const [strings] = await load(['vs/base/common/strings'])
const text = 'Hello שלום مرحبا 😀 e\u0301 👩\u200D💻'
const host = document.createElement('div')
host.style.cssText = 'height:200px;width:600px'
document.body.append(host)
const editor = window.monaco.editor.create(host, { value: text, language: 'plaintext', minimap: { enabled: false } })
try {
const model = editor.getModel()
const initial = model.getValue()
model.pushEditOperations([], [{ range: new window.monaco.Range(1, 1, 1, 1), text: '前缀 ' }], () => null)
const edited = model.getValue()
model.undo()
return {
initial,
edited,
restored: model.getValue(),
rtl: ['ascii', 'שלום', 'مرحبا', '😀'].map(value => strings.containsRTL(value)),
emoji: ['ascii', '😀', '🌍', '🫠'].map(value => strings.containsEmoji(value)),
imprecise: [0x41, 0x1F600, 0x1FAE0].map(value => strings.isEmojiImprecise(value)),
grapheme: [0x41, 0x0D, 0x0A, 0x0301, 0x200D, 0x1F1E6, 0x1F469].map(value => strings.getGraphemeBreakType(value)),
breaks: [[0x0D, 0x0A], [0x41, 0x0301], [0x41, 0x42], [0x200D, 0x1F469]].map(([a, b]) => strings.breakBetweenGraphemeBreakType(strings.getGraphemeBreakType(a), strings.getGraphemeBreakType(b))),
}
}
finally {
const model = editor.getModel()
editor.dispose()
model.dispose()
host.remove()
}
})
expect(observed.initial).toBe('Hello שלום مرحبا 😀 e\u0301 👩\u200D💻')
expect(observed.edited).toBe(`前缀 ${observed.initial}`)
expect(observed.restored).toBe(observed.initial)
expect(observed.rtl).toEqual([false, true, true, false])
// Keep the historical approximate predicate; this is not a current Unicode conformance test.
expect(observed.emoji).toEqual([false, true, true, false])
expect(observed.imprecise).toEqual([false, true, false])
expect(observed.grapheme).toEqual([0, 2, 3, 5, 13, 6, 14])
expect(observed.breaks).toEqual([false, false, true, false])
await testInfo.attach('monaco-unicode-observations', { contentType: 'application/json', body: JSON.stringify(observed) })
})
+1
View File
@@ -45,6 +45,7 @@ test('mobile vConsole shows logs and upstream site notice texts are served uncha
expect(await index.text()).toContain('Included component: @vscode/codicons 0.0.26')
expect(await index.text()).toContain('Included component: typescript (Monaco worker) 4.4.4')
expect(await index.text()).toContain('Included component: dompurify (Monaco core) 2.3.1')
expect(await index.text()).toContain('Included component: Unicode data (Monaco core) mixed historical inputs: 10.0.0d5, 13.0 drafts, Emoji 13.1')
expect(await index.text()).toContain('Included component: marked (Monaco core) 3.0.2')
expect(await index.text()).toContain('Included component: vscode-html-languageservice 4.1.1')
expect(await index.text()).toContain('Included component: vscode-json-languageservice 4.1.9')
+63
View File
@@ -0,0 +1,63 @@
import assert from 'node:assert/strict'
import { Buffer } from 'node:buffer'
import fs from 'node:fs'
// eslint-disable-next-line test/no-import-node-test -- Exercise frozen Unicode recipes and source boundaries.
import test from 'node:test'
import { verifyMonacoUnicodeNotices } from '../scripts/site-vendor/monaco/unicode-origins.ts'
import { readUnicodeTables, replayUnicodeGenerator, verifyUnicodeGeneration } from '../scripts/site-vendor/monaco/unicode.ts'
const source = 'const CONTAINS_RTL = /a/; const CONTAINS_EMOJI = /b/; export function isEmojiImprecise(x: number) { return (x === 1); } function getGraphemeBreakRawData() { return JSON.parse("[0,0,0]"); }'
const output = logs => ({ logs, writes: new Map() })
const rtl = output(['------', 'a', '------'])
const emoji = output(['very imprecise test :: x === 1', '------', 'b', '------'])
const tree = output(['[0,0,0]'])
test('Unicode extraction identifies exact top-level declarations and literal data', () => {
const tables = readUnicodeTables(source)
assert.deepEqual(tables, { rtl: 'a', emoji: 'b', imprecise: 'x === 1', grapheme: [0, 0, 0] })
assert.deepEqual(readUnicodeTables(`${source}\nconst text = 'CONTAINS_RTL = /wrong/;';`), tables)
assert.deepEqual(readUnicodeTables(source.replace('x === 1', 'x\n /* trivia */ === 1')), tables)
assert.throws(() => readUnicodeTables(`${source} const CONTAINS_RTL = /c/;`), /repeated Unicode/)
assert.throws(() => readUnicodeTables(source.replace('const CONTAINS_RTL = /a/;', '')), /Missing Unicode/)
assert.throws(() => readUnicodeTables(source.replace('/a/', 'getRegex()')), /regex literal/)
assert.throws(() => readUnicodeTables(source.replace('/a/', '/a/g')), /regex flags/)
assert.throws(() => readUnicodeTables(source.replace('JSON.parse("[0,0,0]")', 'getTree()')), /literal Unicode tree/)
assert.throws(() => readUnicodeTables(source.replace('[0,0,0]', '[0,0]')), /tree integers/)
assert.throws(() => readUnicodeTables(source.replace('[0,0,0]', '[0,-1,0]')), /tree integers/)
})
test('Unicode comparison rejects dataset substitutions and incomplete generator output', () => {
const tables = readUnicodeTables(source)
verifyUnicodeGeneration(tables, rtl, emoji, tree)
assert.throws(() => verifyUnicodeGeneration({ ...tables, rtl: 'c' }, rtl, emoji, tree), /RTL differs/)
assert.throws(() => verifyUnicodeGeneration({ ...tables, emoji: 'c' }, rtl, emoji, tree), /emoji differs/)
assert.throws(() => verifyUnicodeGeneration({ ...tables, imprecise: 'x === 2' }, rtl, emoji, tree), /imprecise emoji differs/)
assert.throws(() => verifyUnicodeGeneration(tables, rtl, emoji, output(['[0,1,0]'])), /grapheme tree differs/)
assert.throws(() => verifyUnicodeGeneration(tables, output(['a']), emoji, tree), /regex delimiters/)
assert.throws(() => verifyUnicodeGeneration(tables, rtl, output(['------', 'b', '------']), tree), /imprecise emoji expression/)
})
test('Unicode recipes only read declared inputs and capture declared writes without filesystem output', () => {
const inputs = new Map([['./data.txt', Buffer.from('frozen')]])
const recipe = 'const fs = require("fs"); fs.writeFileSync("out.txt", fs.readFileSync("./data.txt").toString("utf8")); console.log("result");'
const actual = replayUnicodeGenerator(recipe, inputs, ['out.txt'], {})
assert.deepEqual(actual, { logs: ['result'], writes: new Map([['out.txt', 'frozen']]) })
assert.throws(() => replayUnicodeGenerator(recipe.replace('./data.txt', '../outside'), inputs, ['out.txt'], {}), /Unexpected Unicode input/)
assert.throws(() => replayUnicodeGenerator(recipe.replace('out.txt', '../outside'), inputs, ['out.txt'], {}), /Unexpected or repeated Unicode output/)
assert.throws(() => replayUnicodeGenerator('require("node:fs")', new Map(), [], {}), /Unexpected Unicode import/)
assert.throws(() => replayUnicodeGenerator('', inputs, [], {}), /Unused Unicode input/)
assert.throws(() => replayUnicodeGenerator('', new Map(), ['out.txt'], {}), /Missing Unicode output/)
assert.throws(() => replayUnicodeGenerator('require("fs").writeFileSync("out.txt", "a"); require("fs").writeFileSync("out.txt", "b");', new Map(), ['out.txt'], {}), /repeated Unicode output/)
})
test('Unicode notices bind complete data terms and attribution to editor and worker', () => {
const manifest = JSON.parse(fs.readFileSync('scripts/site-vendor/manifest.json', 'utf8'))
verifyMonacoUnicodeNotices(process.cwd(), manifest)
const group = manifest.groups.find(item => item.name === 'monaco-editor')
const component = group.components.find(item => item.name === 'Unicode data (Monaco core)')
component.assets.pop()
assert.throws(() => verifyMonacoUnicodeNotices(process.cwd(), manifest), /notice binding/)
component.assets.push('docs/assets/js/vs/base/worker/workerMain.js')
group.notices = group.notices.filter(item => !item.target.endsWith('core-unicode/LICENSE.txt'))
assert.throws(() => verifyMonacoUnicodeNotices(process.cwd(), manifest), /Missing Monaco Unicode notice/)
})