Notes › Tech Notes › Binary and hex conversions keep going wrong: base 2, 8, 10 and 16

Binary and hex conversions keep going wrong: base 2, 8, 10 and 16

Last month I was debugging an industrial PC that kept raising alarms. The line in the log read alarm=0x0E12. I glanced at it, read E12 as decimal and got three thousand six hundred and twelve, then took the bit definition table from the manual and counted from bit0 to bit12, matched it to a fault that does not exist, and spent most of a day investigating in the wrong direction. When I finally expanded 0x0E12 into 0000 1110 0001 0010 the picture was clear: the bits set to 1 were bit1, bit4, bit9, bit10 and bit11, which add up to 2+16+512+1024+2048 = 3602. The gap to 3612 was me treating the hex digit 1 as if it sat in the tens place. The value never changed, only the notation did.

1. A base is just another way to write the same number

Once a value is fixed, how many symbols you use to write it is up to you. The radix decides how many symbols each digit can hold and what one carry multiplies by.

BaseRadixDigitsPrefix in codeWhere you meet it
Binary20 10bBitwise operations, registers, masks
Octal80-70o (older style: leading 0)Linux permissions, chmod 755
Decimal100-9NoneEveryday values, counters in logs
Hexadecimal160-9 A-F0xColors, memory addresses, protocol byte streams

Hexadecimal is everywhere in engineering because it lines up with binary in groups of four: one hex digit covers exactly four bits, so a glance at 0xFF tells you it is eight bits all set.

2. Three moves cover manual conversion

  • Decimal to another base: divide by the radix repeatedly and keep the remainders, then read them backwards. Take 3756 to hex: 3756 / 16 = 234 remainder 12 (C), 234 / 16 = 14 remainder 10 (A), 14 / 16 = 0 remainder 14 (E), which read backwards gives 0xEAC.
  • Another base to decimal: expand by positional weight. EAC = 14x256 + 10x16 + 12 = 3756, and getting back to the original value means the previous step was right.
  • Binary and hex: cut four bits at a time from the right and look each group up. 3756 in binary is 1110 1010 1100, and the three groups are E, A and C.

While I was at it I checked a rarer one: 12345678 in base 36 (0-9 plus A-Z) is 7CLZI. Expanding by weight, 7x36⁴ + 12x36³ + 21x36² + 35x36 + 18 = 11757312 + 559872 + 27216 + 1260 + 18 = 12345678, back to the same value. Short links and order suffixes, those mixed letter-and-digit codes, are almost all base 36.

3. The numbers you meet most, worth memorising

DecimalBinaryHexadecimalOctalWhere it shows up
2551111 1111FF377Upper bound for one RGB channel, eight bits set
4201101001001A4644The real value behind chmod 644
4931111011011ED755The real value behind chmod 755
40961 0000 0000 0000 0000100010000Bytes in one memory page
6553516 onesFFFF177777Port number ceiling, unsigned 16-bit
1677721524 onesFFFFFF77777777White in 24-bit color, #FFFFFF
214748364731 ones7FFFFFFF17777777777Signed 32-bit ceiling, the error value when an API overflows
429496729532 onesFFFFFFFF37777777777Unsigned 32-bit, IPv4 broadcast address

One practical takeaway from the octal column: 755 and 493 are the same permission written two ways. Storing either in a config file is fine; mixing them is what hurts. I have seen one script write 755 in one place and 493 in another, and the diff makes it look like two separate changes.

4. What I actually do now

  1. Copy the original notation from the log verbatim and keep the 0x and 0b prefixes. Do not retype from memory; half of accidents like the E12 one come from hand copying.
  2. For the value itself I use Number Base Converter: pick the source base and it shows binary, octal, decimal and hex together, which saves me aligning digits while punching a calculator. When I need to tick bits in a mask one by one, I watch the binary output, not the hex.
  3. When a byte stream has to be read character by character I use the other direction: text becomes a base encoding per character, so A1 gives 41 31, with a choice of separator, which is how I check a packet capture against a protocol table byte by byte.
  4. Six-digit hex colors I no longer split by hand. I used to break #FF8A00 into three parts and divide by 255, and I kept dropping carries; using Color Converter to line HEX up with RGB and HSL is easier.

If all you want is to normalise full-width and half-width characters or read ASCII values, Text Encoder is the better fit; it works at the character level, not at the numeric base.

5. Three easy traps

Prefix and default radix disagree. In JavaScript, parseInt('0755') gives 755, and only parseInt('0755', 8) gives 493. The literal 0755 throws a SyntaxError outright in strict mode and ES modules, so octal has to be written 0o755. One permission value, three notations, three outcomes, so always pass the radix when config crosses languages.

Large numbers lose precision before the base even matters. A double can represent integers exactly only up to 2⁵³−1 = 9007199254740991, and beyond that 9007199254740993 is stored as ...992. Before converting such a value, confirm the tool works through BigInt, otherwise the hex looks fine while the tail digits are already wrong.

Negative numbers have no standard hex. Writing -1 as FFFFFFFF only holds under 32-bit two's complement; in 16 bits it is FFFF. A base conversion handles the value, and you have to bring the bit width yourself.

6. Common questions

Q: Why do computers have to use binary?

Two physical states are the most reliable (high and low), with the best noise immunity and the cheapest implementation. Octal and hex are not for the machine; they compress dozens of 0s and 1s into something the human eye can scan.

Q: How do you convert a hexadecimal fraction?

Integers divide by the radix and keep remainders; fractions multiply by the radix and keep the integer part: 0.5 x 16 = 8.0, so 0.5 is written 0.8. But 0.1 is infinitely repeating in any base built on 2, which is one source of floating point error, so do not expect another base to make it go away.

Q: Is 0xFF different from the string "FF"?

Yes. The first is a numeric literal, the second is text, and it only becomes 255 if you parse it explicitly as base 16. Nearly all hex in logs, JSON and config files is in string form, which is where a lot of conversion mix-ups start.

Q: Why does base 36 stop there?

36 is exactly ten digits plus 26 letters. Going higher means distinguishing upper and lower case (base 62) or adding other symbols, and both readability and case-insensitive settings fall apart.

Q: How do you check whether a bit mask has one bit too many?

Convert the total back to binary and line it up against the table in the manual, and confirm once which index the two sides start from, 0 or 1. What I do is expand 3602 into 0000 1110 0001 0010 and tick it against the table, which makes the extra bit easier to spot than adding the bit values in reverse.

The machine reporting 0x0E12 turned out to have an encoder fault and an overheated motor triggering at the same time; the base had nothing to do with it. It did break one habit of mine: any value with a 0x or 0o prefix now gets expanded into binary or decimal in a comment, instead of being converted in my head. Human decimal intuition is the one link that fails.

📎 Related articles

PNG compressed three times only lost 426 bytes: the real boundary of lossless compression 2026-10-22 How Many Times Bigger a PNG Gets After Converting From WebP: When It Is Unavoidable and How to Keep the Growth Down 2026-10-22 Three Reasons the Small Text Smudges When a Screenshot Is Shrunk to a Quarter 2026-10-22 Cropped to the Same 300x300, Why Does the Avatar Always Sit Half a Cell Low-Right 2026-10-22 Forecasting 18 months at 1.7 times a month: why the script and the sheet give two numbers 2026-10-22