For a while we used MD5 to verify installer packages. One day a colleague said the hash did not match and the package was bad, so the whole team went through the CDN, and the cause turned out to be that he had downloaded a 404 page served by the CDN - a few hundred bytes of HTML, so of course the hash was different. Another case was worse: an old system stored user passwords as plain MD5, and less than two days after a database leak someone had restored weak passwords from an off-the-shelf collision table. Those two episodes made me separate what a hash is for from which algorithm to use: checking whether a transfer got corrupted and preventing someone from forging content are two completely different requirements.
1. Three properties of a hash decide the three jobs it can do
- Fixed length: whether the input is 3 bytes or 3 GB, SHA-256 always outputs 256 bits, or 64 hex characters.
- One-way: you cannot get the original back from the digest. This is fundamentally different from encryption, which is reversible with a key and exists to keep things secret; a hash exists to be a fingerprint.
- Avalanche effect: change one character of the input and about half the bits in the digest change. The MD5 of an empty string is
d41d8cd98f00b204e9800998ecf8427eand the SHA-256 ise3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855, while adding a single trailing space produces two completely unrelated strings. Those two fixed values are useful for self-checks: if your result does not match, something is wrong with the tool or the input.
So it can reliably answer three questions: are these two pieces of content the same; has this content been modified; and can I prove I know the original without revealing it. What it cannot answer is what a given ciphertext says.
2. How broken MD5 is, and what it is still good for
| Algorithm | Output | Hex length | Status | Suitable uses |
|---|---|---|---|---|
| MD5 | 128 bits | 32 characters | Collisions can be constructed on ordinary hardware; declared broken for signed certificates | Integrity checks in non-adversarial settings, bucketing for deduplication |
| SHA-1 | 160 bits | 40 characters | Real collisions exist; mainstream browsers and certificate authorities have retired it | Checksums for legacy systems |
| SHA-256 | 256 bits | 64 characters | Currently recommended, widely used for signatures, block digests and artefact verification | The default for security-sensitive work |
| SHA-512 | 512 bits | 128 characters | Stronger, but longer digests raise transfer and manual comparison costs | High-strength requirements, where a longer digest is wanted |
A collision means someone deliberately constructs two inputs with different content and the same digest. Once that can be manufactured, the premise that equal digests mean trustworthy content collapses - which is why MD5 was pushed out of security work. Look the other way, though, and an interrupted download, a CDN serving the wrong origin object or a file mangled by a script are not someone constructing collisions, and MD5 is still fine. The real dividing line is one question: is there an adversary who will actively construct a second piece of content.
3. Password storage is a third job; do not mix it in
Using MD5 or SHA-256 to store passwords is the most expensive misuse in this family. A fast hash is exactly what an attacker wants: one consumer graphics card tries vast numbers of candidates per second, and with a credential-stuffing script and a ready-made rainbow table, common weak passwords fall almost instantly. The right answer is a salted slow hash - bcrypt, scrypt or Argon2id, which carry their own salt and let you tune the work factor so every verification costs a controlled amount of computation. The salt makes identical passwords produce different digests, which defeats bulk precomputation; it is not a key and can be stored in plain text.
4. What I actually do
- Check the algorithm name and format before comparing anything. Many mismatches come from mixing 16-character and 32-character forms: a so-called 16-character MD5 is just characters 9 to 24 of the 32-character result, so it can never match the full value; likewise for case, where
a–fandA–Fexpress the same digest. - For text, compute everything in one tool in a single pass, then compare. I paste the content into the Hash Calculator, which lists MD5, SHA-1, SHA-224, SHA-256, SHA-384 and SHA-512 at once; I compare line by line against the value published by the other side and copy just the line I need. Many legacy systems only publish MD5 while new specs ask for SHA-256, and getting both in one pass saves switching tools and recomputing; normalising case while comparing removes half the false mismatches.
- Files go through file hashing. For something like an installer a few hundred MB in size, use File Hash Calculator, which takes several files at once and returns MD5, SHA-1 and SHA-256, so checking whether a download went wrong is immediate.
- Sign APIs with HMAC, not a bare hash with a key glued on. Constructions like
md5(secret + params)have been bypassed before, through length extension and similar techniques; HMAC-SHA256 hands the key's involvement to a structure designed for it. The same calculator produces an HMAC value once you enter the key, which is what you compare against a client signature.
5. Common questions
Q: What is the actual difference between hashing and encryption?
A hash is one-way, fixed length and needs no key, and is used for verification and fingerprints; encryption is reversible and needs a key, and is used to keep data secret in transit and at rest. If you need the content back, only encryption will do - there is no decrypting a hash.
Q: Can SHA-256 be reversed to the original?
No. The only route is trying candidates and comparing digests, which means the randomness of the original decides its security: a password that is too short gets cracked just as easily after switching to SHA-256.
Q: A file checksum does not match; what should I suspect first?
Suspect that you are not comparing the same thing: an error page or a truncated download, altered line endings in text (CRLF against LF changes a plain-text digest completely), an encoding switch from UTF-8 to GBK, or a 16-character value compared against a 32-character one. A quick look at file size and line count rules out about eighty percent of false alarms.
Q: Can I invent my own hashing scheme?
Not advisable. Security comes from a public algorithm surviving years of attack, not from concatenating digests or defining a custom ordering, and those arrangements tend to introduce new constructible points. Choose SHA-256 or SHA-512, add HMAC, and use an established password hashing algorithm; that covers the overwhelming majority of needs.
Q: Do deduplication and duplicate-file hunting need SHA-256 too?
With no adversary involved, MD5 is faster and simpler, since all you need is whether two pieces of content are identical. Only where tampering is a real concern is it worth moving up to SHA-256, and then the trustworthiness of the channel that distributes the digest matters just as much.
The conclusion from those two incidents is plain: state up front whether this comparison is to catch transfer errors or to stop deliberate forgery, and the algorithm choice settles itself. And when something does not match, the first move should be confirming that both sides hashed the same bytes, not suspecting that the algorithm has been broken.