The Content Addressing Revolution: How CIDs Are Transforming Data Integrity

The address that tells you nothing

For thirty years we have named data by where it sits: a URL, a file path, a row id. Every one of those names shares a flaw that is easy to state and unpleasant to live with. The address says nothing about what you will find there.

https://lab.example/data/final_v2.csv is a claim about a server's filesystem, not about bytes. Overwrite the file and the address still resolves. Delete it and the address still looks valid right up until someone clicks it. Cite it in a paper and you have cited a location, and locations are administered by people who reorganize directories.

Content addressing inverts the relationship. The name is derived from the bytes, so it cannot outlive them unchanged.

What a CID actually is

A content identifier is a hash of the data, wrapped in enough self-description to say which hash function produced it and how the bytes should be interpreted. That self-description is why CIDs survive algorithm changes: the string carries its own decoding instructions rather than assuming SHA-256 forever.

The construction is short enough to read in one sitting:

import { CID } from 'multiformats/cid';
import { sha256 } from 'multiformats/hashes/sha2';
import * as raw from 'multiformats/codecs/raw';

async function cidFor(bytes: Uint8Array): Promise<CID> {
  const digest = await sha256.digest(bytes);
  return CID.create(1, raw.code, digest);
}

const cid = await cidFor(new TextEncoder().encode('Hello, Research!'));
console.log(cid.toString());
// bafkreid2appi7dgxybpite5mlbtz65lb7t6bfksdjqal4q7edbk3qv4wjy

Run it. You will get that exact string, because it is not an illustration - it is what those sixteen bytes hash to under CIDv1 with the raw codec. Every CID printed in this post was computed the same way, and you can check any of them without asking us for anything.

One digit

Tamper evidence is the property people find easiest to believe and hardest to picture, so here it is at full width:

Temperature: 23.5°C
  bafkreigvbdydplcyarvmskvlevkmbb2da3efrfr7xogtc4ezceqkxtvd3m

Temperature: 25.5°C
  bafkreidoxf5pqq3dumvovph4valimq5f6f4bcodhtxxru2kh2cykvdyrku

One character changed in the reading. The two identifiers share the prefix bafkrei, which is just the version and codec announcing themselves, and agree on nothing after it. There is no partial match and no near miss - a falsified measurement does not produce a nearly-correct name, it produces an unrelated one.

This is what makes a CID citable in a way a URL is not. When a methods section references a CID, it references those bytes and no others.

Deduplication falls out for free

Nobody designs for it, but identical content converging on one identifier is a direct consequence of the naming scheme. Three labs that independently archive the same reference genome store one copy between them, and none of them had to coordinate to make that happen.

Structure, not just blobs

IPLD extends the same idea to linked data. A node can reference another node by its CID, and because that reference is itself content-derived, the hash of the parent depends on the full contents of everything it points at. Change a leaf and every identifier on the path to the root changes with it.

That property is what lets a dataset, its processing code, and its results share one verifiable structure instead of three loosely-associated downloads.

What content addressing does not give you

Here is the part that usually gets left out of posts like this one.

A CID is a guarantee about integrity, not about availability. It says: if you obtain bytes under this name, you can prove they are the right ones. It says nothing whatsoever about whether anyone still has them. A CID for data that no node stores is a permanently valid name for something you cannot retrieve - the identifier does not rot, but the bits are just as gone.

So content addressing converts "is this the data the paper used?" from a question you have to trust someone to answer into one you can settle yourself. It does not convert "can I get the data?" into anything at all. That still requires pinning, replication, and somebody paying for disks - which is why the storage layer underneath matters, and why "we use IPFS" is not by itself a preservation strategy.

Being precise about this is worth more than the stronger claim would be. An integrity guarantee you can rely on absolutely is more useful than an availability guarantee you have to qualify every time you make it.

The last row is the honest one, and it is the row that decides your architecture.

Why this is the foundation and not the building

Content addressing is a small idea. Name things by what they are rather than where they are, and a category of failure - the silent substitution - stops being possible. That is the whole of it.

What it buys is narrow and absolute, and narrow-and-absolute composes well. Every layer above it, from provenance graphs to signed credentials to reproducible pipelines, gets to assume that a reference means one specific sequence of bytes. Without that assumption each of those layers would need its own integrity story. With it, they need none.

Further reading: CID specification, IPLD documentation, Multiformats