Involves

Moment

The paper that measured surprise

Bell Labs, 1948; Claude Shannon turns uncertainty into bits

In 1948, Bell Labs researcher Claude Shannon published “A Mathematical Theory of Communication.” He treated a message not by what it meant, but by which possibility had been selected. When only one outcome is possible, uncertainty is zero and no information is gained by learning the result. More equally likely possibilities require more bits to distinguish.

Author
Claude Shannon
Published
1948
Unit
The bit

The caveat

Shannon deliberately set meaning aside. His measure tracks uncertainty in a source, not importance, truth, or understanding. A random string can carry many bits while meaning nothing to its reader.

The paper established limits for compression and reliable communication through noise. Those limits guide digital storage, networks, and error-correcting codes. Its connection to tautology is conceptual: a tautology is certain in every case, while Shannon's framework assigns zero uncertainty to a source with only one possible outcome.

“Bit,” short for “binary digit,” was suggested by mathematician John Tukey. Shannon made it the standard unit for choosing among alternatives.

How it connects

Shannon made the same trade quantitative: when only one outcome is possible, learning the result adds no information.

This moment appeared in When certainty says nothing, the Involves connection for August 6, 2026, which asked: What can a statement tell you if nothing could prove it wrong?

Check yourself

In Shannon's 1948 framework, when does learning a result add no information?

When only one outcome is possible, so uncertainty is already zero.. Right. He measured a message by which possibility had been selected. One possible outcome means zero uncertainty and zero information gained.

What the sources establish

  • In 1948 Shannon published “A Mathematical Theory of Communication,” measuring information by the selection among possible messages rather than by meaning.

  • Shannon adopted “bit,” short for binary digit, as the unit when information is measured with base-2 logarithms, crediting the word to J. W. Tukey.

  • Shannon treated semantic aspects of communication as irrelevant to the engineering problem of transmitting any one message selected from a set.

Sources