Pasting All 120 Chapters of Dream of the Red Chamber Into a Character Frequency Counter: Why Rank 1 Is the Comma at 59,260 Hits

Pasting a Whole 120-Chapter Book Into the Text Frequency Counter

Someone pasted the complete public-domain text of a 120-chapter classic into the character box of the text frequency tool, pressed Enter, and the leader of the board was not a Han character at all but the comma, at 59,260 hits. That TXT measured 2,605,929 bytes and 5,496 lines, one UTF-8 file with no BOM, and non-ASCII bytes made up 99.78% of it.

Bytes, Code Units and Han Characters Are Three Ledgers

The original file for this 120-chapter edition already sits on the site's own resource page, the Dream of the Red Chamber resource page, and its size agrees under two measurements, wc -c and fs.statSync, both 2,605,929 bytes. The counter shown on the page reads text.length, which is UTF-16 code units: 872,515, that is 1,733,414 below the byte count, and bytes divided by code units comes to about 3.0. Counting Han characters only, with /[\u4e00-\u9fff]/g, is a third figure again: 732,628. Feed the whole book into the Text Frequency Counter and that box reports 872,515, not the 2.6 MB in the file name. The three numbers keep separate books and none of them can stand in for another.

Why the Three Chapter Counts Differ as 124, 112 and 120

The loose pattern hit 124 times, and that is not 120 chapters: 120 of those 124 are chapter-heading lines (the opening 【 and closing 】 bracket each appear 120 times), 3 more land inside narrative sentences and 1 is a bare repeated heading with no brackets. The strict pattern written on Chinese numerals hit only 112 times, and all 12 missing ones fall after chapter 100, because that stretch writes its numerals with 零: the character class never included it, so the whole block was skipped, and 112 plus 12 closes against 124. The third independent reading is to type all 120 Han chapter headings, one per line, into the keyword box, which returns 124 total hits with 0 missing.

The Comma Leads, and 4 of the Top 10 Are Punctuation

The full board holds 4,454 rows sorted by hit count descending. The best-ranked Han character is 了 at 21,221 hits in rank 3, that is 38,039 behind the comma, and 的 follows at 15,719 in rank 4. The template skips only four whitespace classes, so every punctuation mark competes for a place, and 4 of the top 10 rows turn out to be punctuation. The table below lists the top 10:

The literary function words all sit low: 之 at 2,147 hits (rank 75 of the whole board), 也 6,116, 者 416, 乎 70, 矣 65, 焉 26 and 哉 26. Taken alone, 之 is only 13.7% of 的, and the textbook yardstick of roughly 12.7% density for the letter E comes nowhere near any single row on this board.

Characters Per Chapter and Lines Per Chapter Do Not Move Together

Across the 120 sections the blank-stripped average is 7,211.3 characters per chapter and the Han-only average is 6,105.2. The longest chapter, chapter sixty-two, runs to 11,760 characters; the shortest, chapter twelve, holds 4,215. That is a gap of 7,545 and a ratio of 2.79. Rank the same chapters by lines and the order flips: chapter seventy-eight spans 123 lines yet only 10,653 characters, because verse produces many short lines, while chapter seventy-four takes 25 lines and still reaches 10,956 characters.

Fig. 1: on the character board chapter 74 outranks chapter 78, yet on the line board 78 has 123 lines against 25 for 74, so the two rankings swap places.

English Word Frequency Returns 7 Rows of URL Debris

The word-frequency box splits on /[a-zA-Z]+/g, so a word is a run of ASCII letters: the 732,628 Han characters contribute 0 words, the whole book totals only 8 words, and the default list of 30 stop words eats by, leaving 7 rows at 1 hit each and 12.5% each. Every row is URL debris from the attribution lines at the end: https, zh, wikisource, org, wiki, cc, sa. Tick the case sensitive option and the board becomes 8 rows, because BY escapes the filter; clear the stop words and it is still 8 rows, with shares adding to 100.0% where the default setting only reaches 87.5%. Cut the book up and feed it chapter by chapter, and 119 of the 120 sections simply answer No data.

The Keyword Box Counts Strings, Not People

Keyword counting walks the text with indexOf in non-overlapping order. Three role names typed on three lines: 宝玉 (Baoyu) 3,992, 凤姐 (Feng Jie) 1,751 and 黛玉 (Daiyu) 1,377; the summary reports 7,120 total hits with 0 missing, and the three figures close exactly. But that percentage column divides by the 872,515 code units of the whole text, so Baoyu shows 0.5% while the 7,984 code units it really occupies are 0.9151%, a factor of two between the two boxes. The list is not de-duplicated either: write Daiyu twice and it reports 4 keywords and 8,497 hits, although the text holds only 7,120 such places. The line 林黛玉把花具且都放下 is 10 characters and sits in the burying-the-flowers passage of chapter 23; the line 凤姐儿偏拣了一碗鸽子蛋放在刘姥姥桌上 is 18 characters and sits in chapter 40.

Three Length Definitions Form a Ladder That Closes

Every gap between the three definitions of a character count can be measured against another one: 872,515 code units minus the 5,742 whitespace characters the template skips gives the template count of 866,773. The template does not eat the full-width space, so removing a further 1,412 gives the blank-stripped count of 865,361. Removing 132,733 punctuation marks then gives the Han count of 732,628. The 4,454 rows of the whole board only add up to 81.4% of shares, because the skipped whitespace stays inside the denominator.

Fig. 2: four definitions stepping down from code units to Han characters, each subtraction reproducible.

Whether the Whole Book Fits in a Single Paste

None of the four input fields in the template carries a maxlength attribute: reading .maxLength in the browser returns −1 and getAttribute returns null, and the code holds no truncation statement either. On the no-limit reading, the full text of 872,515 code units fits into the box in one paste, assignment costs 4.7 ms and reading it back gives the same length. Even three times the whole text, 2,617,545 code units, still fits, and the leading row moves to 177,780 hits, exactly triple. The cost is rendering: writing 4,454 rows as one table takes 77.3 ms of JS plus 1,046.7 ms for the first frame, about 1.12 seconds, and the container's scrollHeight is 106,896 pixels against 320 visible pixels.

Chapter Runs and the Whole-Book Run Confirm Each Other

Running character frequency once per section for all 120 sections took 117.7 ms in total, 0.98 ms on average. Each per-chapter board averaged 953.2 rows while the single whole-book run returned 4,454 rows, yet the sum of the per-chapter hit counts is 866,773, identical to the whole-book figure, so cutting the text swallows no characters. Timings for the three modes, measured in Node, are 58.7 to 76.9 ms for character frequency, 3.2 to 7.0 ms for word frequency and 0.7 to 1.1 ms for the three keywords. Comparing the shipped copy with those measurements gave 14 judgements, 6 consistent and 2 in conflict, and both conflicts sit on definitions of density.

FAQ

Q: Does 124 hits mean there are 124 chapters?

No. The 124 counts how often the string 第X回 appears, while 120 is the number of chapter-heading lines. Of the extra 4, three are ordinary narrative wording and one is a bare repeated heading left by the layout. The strict pattern misses 12 only because 零 never entered its character class.

Q: Why is the English word frequency almost empty?

The splitting expression accepts only runs of ASCII letters, so the 732,628 Han characters never reach that board. All the book offers are the 8 words sitting in the attribution lines at the end, and once stop words are filtered 7 rows of debris remain.

Q: Can that percentage be read as keyword density?

No. The denominator is the total code units rather than the space the word occupies, so Baoyu shows 0.5% while its real occupancy is 0.9151%. For how character frequency, word frequency and stop words read on a small text, the earlier note on stop words and CJK counting covers the basics.

Q: Is a comma at rank 1 worth analysing?

It is what the skip list produces, because punctuation is never excluded. For the Han side read 了 at 21,221, and for the literary function words read 之 at 2,147.