Turn encodeAll On and Chinese First Becomes 中: HTML Entity Table Coverage and Decode Tolerance, Measured

The <div> That Should Render Got Swallowed, and Chinese Turned Into a Run of Digits

Neither incident was a case of "escaping was skipped." Once, pasting an old document back into the page let the line meant to show <div> verbatim get eaten by the browser as a real tag; another time, exporting comments from an old database left every Chinese character looking like &amp;#20013;, a 12-character run that reads like a broken character set. After the fixes one rule survives: count how many layers were encoded before touching anything.

This round pulls the htmlEncode, htmlDecode and html2js functions out of the HTML Entity Encode/Decode template word for word and feeds them in one at a time. Three things must be measured: what the full mode outputs, how far down the entity table it reaches, and what a string off by one character becomes.

Switch on encodeAll and Chinese First Goes Through Double Escaping

Unchecked, it only touches those five special characters and passes everything else through, control characters included. Checked, the extra branch turns every non-ASCII UTF-16 unit into a numeric entity, and the ordering is where it trips up: the non-ASCII pass runs before the ampersand escape, so the ampersand inside the entity it just built is encoded once more by the next global pass. So the output for 中 is the literal &amp;#20013;, 12 characters, not the single-layer 中 many assume.

That string is not the entity itself; it is an entity wrapped in another escape layer. CJK code points are all five digits, so in this mode each Chinese character costs a fixed 12 characters. The on-site article HTML Entity Encoding Traps: Quotes and Angle Brackets keeps hitting the case of storing the escaped form in the database and encoding it again at render time, and the root cause is this ordering.

The Entity-Name Table Lists 17 Keys, but the Named Branch Actually Reaches Only 16

The restore table writes 17 keys, but the named-branch regex only matches an ampersand plus pure letters plus a semicolon. The key ' in the table carries a digit and never gets a turn on the named path, since it is restored by the decimal branch instead, so the named table really covers just 16 names. Feeding everything beyond the basic five, one entry at a time:

• &nbsp; decodes to a half-width space, code point 32, since the template literal writes an ordinary space, not the standard's 160;
• &copy;, &reg;, &trade; and &apos; all restore as usual;
• the two dashes, the ellipsis and all four quote directions are matched.

The &nbsp; entry is the easiest to get wrong: after decoding it looks fine, but it drops the "no line break" property. Encode one non-breaking space back through encodeAll and the output is exactly &amp;#160;, and only when the two sides fail to match do you learn 160 was the real code point.

Tolerance List: Off by One Character and the Whole Segment Is Refused, Silently

The common trait is silence: no error, no warning, the symbol stays untouched.

The Three Replacements Run Serially: One Call Can Eat Two Layers

htmlDecode runs the named, decimal and hexadecimal passes in order, and each pass also scans the fresh string the previous pass just built. Feed in < (9 characters) and a single call yields <: the named pass restores the ampersand and builds <, then the decimal pass swallows it into an angle bracket. Compare <, which peels only one layer and stops, because the named pass never re-reads its own output.

A string that starts with a numeric entity fears one extra decode more than a named entity, and the round trip breaks right at that seam: 12 strings that already look like entities are encoded then decoded, 11 return character for character, and the only counterexample is <; the semicolon-less &lt comes back steady.

Figure 1: round-trip identity count and the single counterexample.

The Byte Bill of Double Escaping: Each Extra Round Nets 4 More Characters

The bloat is only on the ampersand inside an entity: each round turns 1 character into 5, netting 4. Starting from a semicolon-less string is cleanest: &lt is 3 characters, encode once &lt is 7, encode twice &lt is 11. The reverse direction is an arithmetic subtraction: a 13-character string like & leaves 9 after one decode, 5 after two and 1 after three, each step peeling one named entity, and the fourth leaves no change. A lone ampersand is the fixed point of decoding.

Figure 2: both the encode direction and the decode direction are arithmetic.

The Stats Row Reports UTF-16 Units, Not Bytes

That stats row reads the JS string length, which is UTF-16 code units. Two input lines, 中文abc and line2, report 12 raw characters, while the same string's true UTF-8 byte count is 16, and one emoji counts as 2 units. The numbering drifts too: with full coverage on, that emoji is split into ��, two segments, because the regex carries no u flag and therefore takes surrogate values rather than code points.

An Out-of-Range Numeric Entity Does Not Error, It Just Silently Hands You the Wrong Character

The decimal branch performs no range check, and parseInt's result goes straight to a constructor that wraps modulo 65536: 😀 decodes to a private-use character at code point 62976, not that emoji, while � decodes to code point 0, a NUL character. No error and no verbatim keep: the length looks right but the content changed. The loop only closes in your own hands, since re-decoding those two surrogate numbers can rebuild the original character, yet a strict parser rejects them as invalid references.

HTML to JS: Three Kinds of Output Measured, but the List Page Copy Names Only Two

The same two-line input (interpolation, a single quote, a backslash, a backtick; 64 UTF-16 units) comes out at 81, 73 and 95 characters across the three types: template mode is shortest, array join longest. The difference is the escape surface: template mode guards the interpolation marker, while array mode does not touch the dollar sign or the braces and does not process carriage returns at all, so a carriage return is carried verbatim into the string literal on the first line. All three sit in the dropdown, yet the copy mentions only concatenation and template.

What to Do When You Hit a Run of Symbols

Count the layers first, then act: copy the literal string into the decode box, decode once, and if a layer remains decode again; stop when only a lone ampersand is left, except for a <-type string that eats two layers in one call. Store the raw characters and encode once only at output, which is why these two incidents stopped recurring.

Common Questions

Q: I checked encode all non-ASCII, so why does the page show a run of &amp;#20013; instead of Chinese?

It is the replacement ordering: the ampersand inside the numeric entity gets encoded once more, so the output is born as a two-layer form.

Q: &nbsp is one semicolon short, why does it not restore, and why no error?

The named branch's regex demands a closing semicolon, so missing it refuses the whole segment and backfills it verbatim.

Q: After decoding, why does the non-breaking space stop holding the line?

This page restores it to a half-width space at code point 32, not the standard's 160, so the space still looks present while the property is gone.

Q: 12 entity-form inputs encoded then decoded, why do not all of them come back?

11 are identical, and the lone counterexample is the numeric-entity-prefixed one: the three replacement passes run serially, so a single call peels two layers in a row.