Last winter I helped someone pull a year of API access logs off an old machine, and they landed as a 37 MB txt file. I copied it to my laptop and double-clicked it; Notepad spun for several seconds before the cursor appeared, and another editor simply asked whether I really wanted to open it. Worse, when we sent it out for reconciliation, the zipped attachment bounced back for exceeding the size limit. That was the day I admitted one thing: the trouble with text files is usually not the content but the size, and "opens", "sends" and "reads clearly" are three problems that need separate answers.
First, work out where it is stuck
My habit now is to pin down exactly where it hangs, because the three places need different fixes.
When the editor hangs, it is memory and layout. The whole file sits in a buffer, and the visible area still has to re-layout hundreds of thousands of characters; this is not the kind of lag that improves if you wait, it re-layouts on every arrow key.
When transfer hangs, it is a channel limit. Here is a detail that cost me: attachments travel as Base64 in email, so the encoded size is about 4/3 of the original, a third larger. Working back from a 25 MB limit, an original above 18 MB is already at the edge; 18 MB encodes to exactly 24 MB. So I now calculate attachments from the encoded number, not the one in the file properties.
When processing hangs, it is because the file is not meant for my eyes alone. Whoever takes over may care about only a few hours, or need to feed it to a script that accepts only small files; splitting is then not about transfer but about keeping each step's input manageable.
The common fix for all three is to cut the file into pieces, but where you put the cuts decides how much grief you save later.
The trade-offs of three splitting modes
In Text Splitter there are only three modes: by line count, by size and by delimiter, and you can also set the filename prefix and numbering rule, then download the result as an archive. I have hit different problems with each, so the conclusion first: I try line count by default, then delimiter, and split by size only when there is no alternative.
| Mode | Where the cut lands | Is the size of each part controlled? | Can it split a character? | What I use it for |
|---|---|---|---|---|
| By line count | Outside newline characters, a fixed N lines per part | Not directly; varies with line length | No, characters stay intact | Logs, line-by-line exports |
| By size | At the point where the target KB/MB is reached | Yes, that is its only advantage | Possibly, see the next section | When an upstream limit is a hard size cap |
| By delimiter | At a specified marker, such as a blank line or a fixed line | No; blocks vary in size | No | Documents divided into chapters, exports with an end marker |
The advantage of line count is that every record stays whole, so the next person does not have to guess whose tail a half line is. That log had 480,000 lines; at 20,000 lines per part it split into exactly 24 parts of roughly 1.5 MB each, and the editor opens them without a wait.
Delimiter splitting suits cases where the semantic block matters more than the line count. I once had an internal export where every record ended with a fixed -- END -- line; splitting on that marker gave parts that were complete sets of records. The downside is that you do not choose the number of parts, and one oversized block leaves the parts unbalanced.
Splitting by size is worth it in one situation only: when the goal is literally that no part exceeds a given number, for example an API with a request body cap. The price is that the cut is not guaranteed to land on a line boundary or even a character boundary.
The real trap when cutting by bytes
Let me state the trap precisely: not every encoding can be broken this way; only encodings where one character takes several bytes carry the risk.
In UTF-8, English letters, digits and common ASCII punctuation take 1 byte each, while common Chinese characters take 3. The character zhong, code point U+4E2D, is stored as the three bytes E4 B8 AD. In GBK a Chinese character takes 2 bytes, and in UTF-16LE even an English letter takes 2. So if the cut lands between E4 B8 and AD, the first file ends with two incomplete bytes, the next begins with a lone byte, and neither side forms a valid character.
Decode those two pieces as UTF-8 and, at best, you see question marks or replacement characters; at worst, the leftover byte joins a byte at the start of the next piece to form a valid but completely unrelated Chinese character, and the content changes quietly in a way the eye rarely catches. Conversely, if the file is pure ASCII from beginning to end, every character takes 1 byte and no byte-wise cut can break one. That is why the fact that I split an English log last time and nothing happened does not carry over to Chinese files.
One more thing often mistaken for truncation: an encoding mismatch. Chinese saved as GBK and read as UTF-8 is mojibake throughout, yet not a single byte is missing; open it with the right encoding and it is fine. To check for this kind of problem I usually run File Info Reader over the parts to see sizes and timestamps in bulk, and confirm no zero-byte half-finished file is hiding among them.
Run a conservation check after splitting
I always compare three things: line count, byte count and the first and last lines.
Line count: the parts should add up to the original line count, assuming you split by line count and the original has no extra blank line at the end. Byte count: the sum usually differs a little, because a newline may be added at a cut or the BOM at the head of the file is kept only in the first part; I allow a small delta and investigate if it exceeds expectations. First and last lines: the first line of the first part and the last line of the last part must match the original exactly.
The newline difference sounds trivial but it wrecks the check: in a 480,000-line file, normalizing two-byte Windows line endings to one byte during the split removes 480,000 bytes, about 0.46 MiB, and your byte-count conservation will never match. So confirm which line ending the original uses before splitting.
The last line of defense is to merge the parts back: run them through Text Merger in filename order, and only when line and byte counts match the original can you say the cuts ate nothing. I have done this three times, and once it caught a missing line at the end.
Two small details you learn the hard way
Pad the sequence numbers. Sort part_1.txt through part_24.txt by name and part_10.txt comes before part_2.txt, because the comparison is character by character. I now always pad to two or three digits, as in part_01.txt.
Keep the encoding the same as before. For Chinese files I stay in UTF-8 the whole way, never switch in the middle and never casually convert before merging; mix two encodings and the second half of the merged file will certainly break.
Common questions
Q: Do I have to split a txt file of tens of MB before I can read it?
If you are the only one scanning for a few keywords, you can get by without splitting. Once someone else has to read it, paste it into a ticket or check it section by section, splitting beats scrolling. My threshold: if two or more people need to read the text, split it.
Q: Should I split by size or by line count?
Look at who consumes the file. If it is people and editors, split by line count, because whole lines matter more than even sizes. If it is an API or storage with a hard cap, split by size, then check each part's tail for a broken Chinese character.
Q: The Chinese turned into question marks after splitting. Did the split break it?
Not necessarily. First check whether the whole file is garbled or only the seams. Whole-file garbling usually means the wrong encoding, so open it with another one; garbling at the seam only, one or two characters, means a byte-wise split cut a multi-byte character. That cannot be repaired, only re-split by line count or delimiter.
Q: Can CSV be split by line count, and what about the header?
Yes, but you have to add the header yourself. After splitting I add the same header line to every part; otherwise the columns lose their meaning from the second part on. Also, if a row contains a quoted field with a line break inside, a line-count split will cut one record in two, so check for multi-line fields first.
Q: If I have to merge it back afterwards, was the whole thing pointless?
No. Splitting keeps each step's input manageable; merging verifies that nothing was lost. Two different purposes. If I have to work on the split files for a long time, I keep the original untouched and treat the parts as copies.
Back to that 37 MB log. It ended up as 480,000 lines split into 24 parts of 20,000 lines, the sequence padded to two digits, the encoding left as UTF-8, and the attachment problem solved along the way: each part is 1.5 MB, well under the limit after encoding. If you ask how I choose a splitting mode, it comes down to one sentence: ask who will read this file next and how, then decide whether the cut lands on a newline, a marker or a byte.