Adding pinyin to Chinese text: conversion with tone marks

A while back my mother asked me to type out the poem Mǐn Nóng with pinyin above it, so my four-year-old niece could read it herself. I went through the editor's pinyin guide character by character and marked the character for plant in the line about spring sowing as zhǒng, when in that line it is the verb to sow and should read zhòng. Having fixed that one, I found I had marked the character for noon in the next line as dàng, when in that line it is simply dāng. Two hours went by, and what I remembered was not a layout trick but the real trap in this job: what a given character actually reads as in this one word, this one sentence, while an automatic tool only knows character frequency, not meaning.

1. The hard part of annotation was never typing, it is choosing the reading

Polyphonic characters make up a fair share of common Chinese characters, and a tool can only guess from word frequency and common collocations. These are the ones I kept getting wrong while making materials for my kid, the set that has to be confirmed by hand:

I corrected that row twice: it has only four readings, chā, chà, chāi and cī, with no chǎi. chāishi, chāiqiǎn and chāishi, the words for errand, dispatch and assignment, all read chāi, and cī survives almost only in cēncī, the word for uneven. Inventing another reading from the look of a character is a mistake that automatic pinyin and I have both made.

Proper nouns are even less guessable. Among surnames, the character normally read dān is shàn, jiě is xiè, céng is zēng, chá is zhā, qū is ōu, pǔ is piáo, and chóu is qiú. Among place names, Bengbu is bèngbù, Bozhou is bózhōu, Jingxing is Jǐngxíng and Shenxian is Shēnxiàn. There is also a split convention here: the 1985 Table of Authorised Readings for Variant-Pronunciation Words in Putonghua set the character in Shenxian to a single reading of shēn, which is how I annotated it, while Xinzhuang in Shanghai is still read Xīnzhuāng by habit; when conventions collide like that you can only pick one and note it. Such names appear extremely rarely in any corpus, so any processing that judges by common words will get them wrong. The only way is to build your own lookup table: I set the readings by hand for every surname and every place name I commonly write, saved them as a two-column table, and check results against it every time.

2. Three rules for writing pinyin that most people follow by feel

The tone mark does not go just anywhere. The rule looks at the main vowel of the final: if there is an a, mark the a; with no a, look for o or e; when i and u are side by side, mark the second one. So hǎi takes the mark on a, jiǔ on u, and guī on i, since ui is a contraction of uei and iu of iou. One line often left out: when the final is a single ü, the mark sits on that ü, as in nǚ, lǜ and jǔ, and there is no version that moves it onto the initial. This is the rule I most easily remember backwards, so I use the three syllables jiǔ, guī and nǚ as my check.

Handling ü is the second frequent error: after j, q, x and y it drops its two dots, written juān, qū and xǔ; after l and n the dots must stay, written lǜ and nǚ. Annotating the word for green as lu instead of lǜ is the most typical error in automatic pinyin. And when the keyboard cannot produce ü, the usual workaround is v, as in lv and nv, which matters especially when filling in filenames, passports and ID forms.

The third trap is tone sandhi. Annotation shows the underlying tone, not the pitch actually spoken: with two third tones in a row the first is pronounced second, yet nǐ hǎo is still written nǐ hǎo; yī and bù become second tone before a fourth tone, so yí yàng and bú shì in speech, but written annotation keeps yī and bù. Neutral tone takes no mark, and māma is written mā ma. For materials aimed at children learning characters, marking the underlying tone is correct; for a recitation script I add a separate line of pronunciation notes alongside.

3. How I put together a usable annotated script

1. Decide the purpose first. Literacy materials annotate character by character, aligned above each one; publication-style pinyin transcription writes words joined and capitalises the first letter of proper nouns, and the two layouts are very different. The second follows the Basic Rules of the Orthography of Hanyu Pinyin (GB/T 16159-2012), which requires words to be written as units, so pīnyīn is one whole.
2. Clean the source text first. Traditional Chinese has to go through a simplified-traditional conversion before annotation, or the processing will not cover it; mixed-in digits, Latin letters and punctuation stay as they are, so they do not get pulled into the pinyin.
3. Run the whole passage through Chinese to Pinyin for a result with tone marks. It gives pinyin and characters side by side, so I can check line by line instead of comparing two documents back and forth. When I need abbreviations or mnemonics, I switch to initials mode, for example to give a project a code name.
4. Check proper nouns against my own lookup table, especially names on lists, place names and archaic characters in lines of poetry.
5. If the downstream system does not accept tone-marked characters, as with some subtitle formats or fonts missing glyphs in older editors, I use Batch Find and Replace to build a set of rules that turn the marked vowels back into plain letters: a, e, i, o, u and ü with four tones each, twenty-four rules in one run, more reliable than editing by hand. Where the text has to become Traditional Chinese for Hong Kong or Taiwan, I handle that separately with Chinese Converter, and do the annotation on the simplified text only.

4. A few more things that get overlooked

Spelling a name and annotating pronunciation are two different jobs, and the reference is GB/T 28039-2011, the rules for spelling Chinese personal names in Hanyu Pinyin letters: surname and given name are written separately, surname first, and each part capitalises only its first letter, so Li Ming. For documents there is often an extra requirement that the surname be in full capitals, and the surname is what changes: Wáng becomes WANG while the given name keeps only its initial capital, giving WANG Ming. Writing the given name as MING too merges the two parts into one lump, and redoing it later is a nuisance, so confirm the convention before you make the English name badge.

The erhua ending is written -r, as in huār and yīr, with no extra syllable added; er is listed separately only when it stands as a syllable of its own. In Beijing colloquial materials this is the easiest place to be inconsistent.

If the text involves privacy, watch where it is processed. For names, phone numbers or medical records, I confirm the conversion runs locally in the browser before it is saved, so a whole list never gets pasted into a service I do not know.

5. Common questions

Q: Can automatic tools get polyphonic characters right?

For common phrases, mostly yes, with high-frequency combinations like chóngfù (repeat) and yínháng (bank); single characters, rare combinations and personal or place names cannot be trusted. Treating the output as a draft and the review as a required step is the only safe approach.

Q: Should annotation show the underlying tone or the spoken one?

Written convention marks the underlying tone; sandhi, whether two third tones in a row or yī and bù changing tone, and neutral tone happen in actual speech. Teaching materials use the underlying tone, while a dubbing or recitation script can carry a separate line of pronunciation notes.

Q: Should pinyin be split per character or joined per word?

For character-learning annotation, aligning one character at a time is fine; formal transcription of running text has to join words and follow GB/T 16159, or readers cannot reconstruct the sentence from the pinyin.

Q: Can Traditional Chinese text be annotated directly?

Convert it to Simplified first. Even one-to-one simplified-traditional mapping has cases where one word has several renderings, and annotating directly tends to lose tone marks on characters the conversion did not cover.

Q: How do I get pinyin initials quickly?

Switch to initials mode and run the whole passage. It is commonly used for internal code names, password mnemonics and abbreviations. Note that a polyphonic character takes its initial with it, so whether Chongqing comes out as cq or zq depends on whether its first character is judged to be chóng or zhòng.

That poem, Mǐn Nóng, ended up in two versions: the underlying tone above each character, with proper nouns and characters whose reading depends on context, like the ones for plant and for noon, checked against my own table, plus a line of pronunciation notes for the parents. My niece reached the second line and read the word for spring sowing as zhòng, because she saw the two characters for sow and seed that I had written by hand next to it. That settled it for me: the value of an annotated script is not whether the pinyin got generated, but whether anyone checked it against the context of the line afterwards.