Five code units gone and the interface never says a word
Start with a small accounting. An old log pasted into the conversion page has 11 code units in the source, the render only reads 6, and the 5 that go missing leave no trace in either the preview or the finished output, while mdWarnings is an empty array. The second thread came from a reader's question: the HTML copied away is a stretch shorter than the source, so who ate it. Only after reading this page's script did I confirm that "characters quietly going missing" is three mechanisms, each silent in its own way, and that the 271 in the title is the extreme value of one of them.
The page under test is the Markdown to HTML converter: a template of 1135 lines, 41,623 bytes and 72 md* functions. The method was to lift each function verbatim into plain Node and let every DOM call take the null branch of mdGet.
The intermediate state is buried inside the text string
This page does not parse and emit at the same time. Inline code, links, images and emphasis are computed one step ahead, and each result is turned into a single "hole" left inside the string, marked by the two constants at L235 and L236. mdStashPut (L323) returns "code point 2 + index + code point 3", the real HTML lives in stash.list, and once the block level is assembled the main chain calls mdUnstash exactly once, at L668, to refill everything.
The index is the length of the list at insertion time, which has nothing to do with document order. For the document that opens with # A **b** `c`, the index sequence inside the string before refill is 2,0,1,3,4,6,5, so the bold marker's 2 is lined up ahead of the inline code's 0.
A 64-round cap that only a hand-built chain can reach
The loop condition of mdUnstash (L333) is "the string still contains code point 2 and the round count has not reached 64". After it exits, L343 deletes whatever holes remain, unconditionally, and nothing between L329 and L345 ever calls mdWarn. Reaching the cap takes a chain of holes wrapped inside holes (each layer = A + the inner hole + B, innermost Z): at depth 64 the round count is exactly 64 and everything expands into 127 code units; from depth 65 the round count locks at 64 and the output locks at 128 code units, lost characters = 2*(depth - 64) - 1, so 65 loses 1, 66 loses 3, 70 loses 11, 100 loses 71 and 200 loses 271, with no message at any point along the way.
Figure 1: chain depth against the number of refill rounds.
This cap has nothing to do with how long the document is. mdSample() is 53 lines / 655 code units and carries 17 holes before refill, yet the round-by-round log holds one line only, "round 0: holes 17 to 0", finished in 1 round with 0 characters deleted; a document of 2,000 paragraphs and 97,778 code units carries 6,000 holes and still needs 1 round, its output is 179,779 code units, and the probe machine takes 58 ms. When holes do not nest, a single round swaps them all, so 64 rounds leave a typical document 63 rounds of headroom.
Out-of-range holes are the other silent path. Fed X洞0-洞7-洞2Y, where the list has only 3 items, hole 7 cannot be resolved so L337 leaves it exactly as it is, round 2 breaks out, then L343 deletes it as 3 characters, and the finished string is the 17 code unit X<b>0</b>--<u>2</u>Y with every legitimate hole refilled as usual.
Normalization eats characters first
mdNormalize (L523-529) deletes code points 0 through 4 at L527: 甲 + code point n + 乙 goes from 3 code units to 2, each of the five hit once, and 5 pasted together turn 7 code units into 2, while U+0005 and U+0006 are not deleted at all. The opening case, where 11 code units read as 6, is eaten at this step; mdWarn has only two call sites on the whole page, L696 and L766, so the message count is 0. The same regex closes a forgery on the side: write 码位 2 0 码位 3 in the body to fake a hole, 9 code units come in, 7 code units leave normalization, the markers are gone and only the 0 in the text is left.
L526 replaces every tab with a constant 4 spaces, and the indent level only recognizes 4: 3 spaces give <ul>, exactly 4 spaces give <pre><code>, 5 spaces fall back to <p> - 甲</p>; a tab always folds into the 4-space level, a space plus a tab lands on the 5-space level, and two tabs' 8 spaces are a paragraph; a child item indented by 4 spaces counts as the next level down, and that account was settled in list numbering and heading anchors.
Warnings stop at 20 entries
Take a table with a 2-column header and 34 rows of 3 columns, each row carrying one cell too many: L766 emits one warning per row, mdWarn is called 34 times, but L498 only stores the first 20 entries, so the array stops at 20 and 14 of them vanish without a sound. The #md-err text is 513 code units, split on ' | ' it yields exactly 20 entries, and the last one is the "table row 20 ..." line, while rows 21 through 34 do not contribute so much as a hyphen. The budget does not accumulate across runs either: L660 resets the array on every render, and five renders in a row return 20,20,20,20,20.
A truth table for the order of the 12 passes
mdInline (L480-495) is a fixed sequence of 12 steps, inline code near the front and the soft line break last. Swap two adjacent steps and only 4 pairs out of 28 inputs show a difference you can see in the output.
| The pair that was swapped | Difference | What you see |
|---|---|---|
| images and links | 1/28 | the parenthesis is eaten by links first, the exclamation mark stays bare outside |
| links and urls | 2/28 | the closing parenthesis is swallowed into the address |
| escapes and emphasis | 1/28 | a backslash leaks into the finished output |
| rawTags and esc | 4/28 | the tag is escaped first and cannot be pulled back |
| the other 7 pairs | 0/28 | this sample group never triggered it, which is not the same as always commutative |
The image case is fed , and the swapped order turns it into !<a href="…">alt <em>x</em></a>; in the \*不是斜体\* case the two backslashes end up stranded on either side of the tag. mdEmphUnderscore (L456) lists CJK characters among the things that "may not be adjacent", so snake_case_word and _斜体_中文 both come out with 0 holes, untouched.
The two copy buttons differ by 106
Render the 66 code unit sample (it contains <b>, quotes, a&b and a link with a title) at the default option. "Copy HTML" (L1051) hands back 135 code units unchanged; "Copy then escape" (L1054) wraps one more layer of entities and hands back 241 code units. The difference reconciles character by character: each & adds 4, each < and each > adds 3, each " adds 5, so this case is 4×7 + 3×8 + 3×8 + 5×6 = 106, and the same expression gives 1,559 → 2,565 on the sample output. Its character set is not the one mdEsc (L265) uses, because mdEsc writes the single quote as ' while the four steps at L1055 leave single quotes alone; adding only this one layer of entities is the job of HTML escaping.
Clear it, reload, and the sample comes back
mdStore (L1101) writes two keys and nothing else: the source text goes into md2html-input, the options into md2html-opts (7 fields = 6 checkboxes plus charset). mdRestore (L1117) falls back to mdSample() in four situations: an empty store, broken JSON in opts, window.localStorage being null, and an empty string in store after a click on "Clear". The fourth is the least intuitive one: an empty string makes raw && raw.length take the false branch, so what reloads is the 655 code unit sample.
Common questions
Q: Why does nothing warn about the characters that go missing?
Warnings have only two sources, the unclosed fence at L696 and the table row with too many columns at L766. L527 deleting control characters, L343 deleting leftover holes and the truncation from entry 21 onward are none of them on the warning path.
Q: How do I tell normalization from refilling?
If the missing count falls on code points 0 through 4 and nothing past U+0005 moves, it is L527. If a whole tail disappears, or exactly 3 characters go at a time, it is a hole deleted by L343.
Q: Will a long document hit the 64 rounds?
Measured, it will not: 97,778 code units and 6,000 holes still take 1 round. Only the hand-made chain of holes wrapped in holes can reach the cap, and once it does the output locks at 128 code units.