A Lost Schrödinger Manuscript Reveals a Surprising Twist in Quantum Physics

For more than 80 years, a manuscript penned by one of the founders of quantum physics has been gathering dust in an Austrian library.

Now brought to light by physics historian Flavio Del Santo of the University of Geneva, the three-page document is the very soul of brevity – but on it is scrawled a long-lost contribution to one of the most contentious debates in quantum mechanics.

And it offers a surprising insight into what Erwin Schrödinger himself believed about the nature of reality.

Rather than the discomfort with quantum randomness he is often portrayed as espousing, Schrödinger believed that there were aspects of the Universe that couldn't be predicted – a belief he held even before the emergence of quantum theory in the 1920s.

"The strongest plea in favor of indeterminism is older than quantum mechanics," Schrödinger wrote in the unpublished manuscript.

In classical physics, there's no such thing as true randomness. Dice may have nicks and flaws that influence how they roll. Computer random-number generators are usually driven by algorithms. Even coin flips are governed by physical forces that, in theory, could be predicted.

This phenomenon is known as determinism, and it plays an important role in our understanding of the physical Universe, stating that every classical behavior is determined by the conditions that precede it, even if we don't know what all those conditions are.

(The absoluteness of classical determinism has been questioned, but it remains a foundational assumption of classical mechanics.)

But quantum mechanics – as is its wont – marches to the beat of a very different drum.

At the quantum scale, the behavior of particles such as photons and electrons can only be predicted in terms of probabilities. We can calculate the likelihood of a particular outcome with extraordinary precision, but we can't definitively predict what an individual particle will do.

This raises a profound question: Is the Universe fundamentally random, or are there hidden factors that determine particle behavior that quantum mechanics hasn't yet accounted for?

These hypothetical hidden factors are known as hidden variables, and they are what they sound like – unknown properties or conditions that, if we did know them, would allow physicists to predict the quantum behavior of individual particles.

In 1932, mathematician John von Neumann – one of the leading authorities on quantum theory's foundational math – published what was widely interpreted as a mathematical proof that such hidden variables were impossible.

His argument carried considerable weight, and for more than a decade, research into hidden-variable models largely stalled.

Not everyone was convinced. In 1935, mathematician Grete Hermann poked significant holes in von Neumann's reasoning. However, her critique was mostly ignored, and you can probably guess why without much difficulty.

By the mid-1940s, however, two prominent physicists had inadvertently revived the argument for hidden variables – by, ironically, speaking out against them, in agreement with von Neumann.

In May 1943, mathematician and physicist Edmund Taylor Whittaker delivered a lecture arguing that quantum mechanics had finally demolished the classical notion of determinism.

Unlike the apparent randomness of a coin toss, he argued, quantum randomness was genuine, and no amount of additional information could make individual outcomes predictable.

Then, in January 1944, physicist Satyendra Nath Bose made a similar argument in an address to the Indian Science Congress, declaring that "physicists have gained knowledge but lost their faith" in the deterministic laws of nature.

Well. That was a spark to dry tinder.

In July 1944, mathematician Henry Piaggio published an article in Nature titled "Collapse of Determinism," reporting on Bose's arguments. It prompted a flurry of responses from physicists who weren't nearly so convinced that determinism was dead and buried.

Over the ensuing months, the argument blossomed across the journal's pages, with researchers picking apart the supposed impossibility of hidden variables – including Whittaker's seemingly straightforward example involving photons and polarized light.

In his 1943 lecture, Whittaker had proposed a seemingly simple demonstration of why hidden variables couldn't exist, using photons – individual particles of light – passing through a polarizer, a device that filters light according to its orientation.

Quantum mechanics can predict what proportion of photons will pass through the polarizer at any given angle.

But Whittaker argued that if each photon carried a hidden instruction determining whether it would pass through, changing the angle of the polarizer while the photons were in flight would render that instruction useless.

Therefore, he concluded, the outcome couldn't be predetermined by hidden variables.

Enter Schrödinger. Or rather, stay in the wings, Schrödinger.

The physicist had been following the debate, and sometime after November 1944, he penned a response of his own, also titled "Collapse of Determinism" (there were several articles published as part of the debate that used the same title).

In it, he identified a significant flaw in Whittaker's argument.

Whittaker assumed each photon carried a hidden variable with only two possible values, one determining whether it would pass through the polarizer and the other whether it would be reflected.

But Schrödinger pointed out that there was no reason to restrict the hidden variable to just two values. What if it could take any value between zero and one?

He then constructed a mathematical model in which the outcome depended on both the photon's hidden variable and the polarizer's angle.

With the right rules, the model reproduced the probabilities quantum mechanics predicts, without requiring each individual interaction to be random.

There was a second part of the model that was also important. Schrödinger devised a deterministic rule for updating the hidden variable after a photon has passed through the polarizer – which meant his model could be used to reproduce the predictions for subsequent measurements.

Importantly, his aim was not to defend determinism because he necessarily believed in it. He was defending the possibility of determinism against specific claims that it was impossible.

"Experimental evidence + logical deduction can never prove that either the deterministic or the indeterministic view is impossible," he wrote.

Schrödinger's response, however, never saw the light of day.

Why he never published it remains a mystery, although Del Santo has a possible explanation.

In 1944, physicist Heinrich Pelzer published a similar hidden-variable model, showing that Whittaker's argument didn't hold up. Schrödinger may have learned of Pelzer's work and decided that the point had already been made.

Yet Schrödinger's model went further, providing an explicit mathematical rule for how a photon's hidden variable would change after passing through a polarizer, allowing it to account for successive measurements.

Whatever his reasons, the manuscript remained unpublished, preserved in an Austrian archive until Del Santo brought it to light more than eight decades later.

But there's another twist in the tale.

Although Schrödinger is often portrayed as a champion of determinism, his own philosophical position was rather different.

As early as the 1920s, before modern quantum mechanics had even emerged, Schrödinger had argued that the laws of nature might be fundamentally statistical, rather than governed by an underlying clockwork of perfectly predictable events.

Influenced by physicist Franz Exner, he even described classical determinism as a "philosophic prejudice" and an "a priori belief".

And his unpublished manuscript suggests that his position hadn't fundamentally changed by the 1940s – nor did it later.

Although the "Collapse of Determinism" flurry of the 1940s is largely forgotten today, physicists remain divided by its core positions.

Perhaps what we can learn from Schrödinger's belated contribution is to keep an open mind. In a rather fitting twist, the man who gave us Schrödinger's cat has left us with a question that remains, in its own way, both alive and dead: Is the Universe deterministic… or isn't it?

"Schrödinger … had developed a remarkably strong commitment to indeterminism even before, and independently of, quantum mechanics, which he maintained throughout his career," Del Santo writes.

Models such as Schrödinger's, Del Santo concludes, "serve as proofs of principle against widespread beliefs or alleged impossibility claims, as strikingly illustrated by the case of Schrödinger reconstructed here, but never as expressions of a dogmatic commitment to determinism."

His paper is available on preprint server arXiv.