If you find quantum physics hard to understand (or accept), rest assured that you are not alone. Even many physicists (including Albert Einstein, one of its founding fathers) refused to acknowledge that our world can behave so strangely. That atoms or electrons can be at two places at once or that it does not always make sense to talk about properties of particles before they are measured.
Physicists are also only people. When a new theory challenges our worldview, we start to look for mistakes in said theory and not in our assumptions about the universe. Many physical theories had to fight against ingrained beliefs — heliocentric model of the Solar system, Einstein’s special and general relativity, or quantum mechanics are just a few examples. Eventually, the theory prevails, at least until it is replaced by a new, more precise one.
In quantum physics, the assumptions of locality and realism are challenged. The locality assumption — which comes from special relativity — tells us that all information can travel through the universe only at the speed of light and not faster. When we assume realism, we assume that the outcome of a measurement exists already before the measurement is performed. In other words, if I look at the thermometer to find out how warm the weather is today, the temperature is not decided the moment I look at the thermometer; the air has this temperature independent of me looking at the thermometer.
Let’s illustrate that on an example. Suppose I have two coins with a very peculiar link between them. Every time I flip these coins, I get opposite outcomes. If the first coin shows heads, the other one shows tails and vice versa. I never know which coin will give which outcome but whenever I look at one coin, I immediately know what the outcome of the toss of the second coin will be.
So far so good. Now, I will take one of the coins and fly to the Moon while leaving you with the second coin here on Earth. If we now flip our coins at the same time, I immediately know what the outcome of your toss is, and you know the outcome of mine.
What does that have to do with local realism? Since we are now about 400 000 km apart, it takes any information about 1.3 seconds to travel between us. My coin thus cannot know what the outcome of your toss is; similarly, your coin knows nothing about my toss. That is what the locality assumption tells us.
How do the coins know what side up to end to always give opposite outcomes? That’s where the realism comes in. In this situation, it tells us that the result of the measurement has existed before the toss and both coins therefore know what outcome the toss is supposed to give.
This is how anyone should expect two such coins to behave. But if the coins obey the laws of quantum mechanics, things are different. We cannot say that the outcome of the toss exists before we actually toss them. (This is actually a matter of interpretation of quantum physics — it is generally assumed that the measurement outcome is decided the moment the measurement is performed.) That’s why some physicists claimed quantum physics must be incomplete — there must be some underlying theory that explains what outcome every single coin flip will give. And such a theory must be local and realistic.1
What should we believe? Local realism or quantum physics? It turns out there is a simple test for that. Suppose that instead of a pair of coins we have two such pairs and I take one coin of each pair to the Moon and keep the other two coins here with you. If we now both flip coins from the same pair, we will always get opposite outcomes. But if we flip coins from different pairs, any combination of outcomes is possible.
While the experiment with a single pair of coins was largely a matter of interpretation, with two pairs of coins do local realism and quantum physics predict different results. All we have to do is toss the coins many times, each of us deciding randomly which of the two coins to toss each round and then writing down which coin we tossed and what the outcome was. Then, we can compare our data and see which of the two theories is right.2
Although the test of local realism is, in principle, rather simple, it is not easy to build an experiment that can confidently decide whether local realism is true or not. There are two main challenges that need to be solved: The first problem is to make sure that the two systems are spatially separated. Here, it is important that the time difference between the measurements is so small that no communication between the two sites at the speed of light is possible. Since the distance over which quantum systems can be reliably transmitted is strongly limited, there are strict requirements on the synchronisation and speed of the experiment.
The second main problem is an efficient measurement. Most experimental tests of local realism are done with single photons but it is extremely difficult to detect those. The efficiency is so low that often detectors do not notice when a photon arrives. This opens a loophole — the measurement does not grant us access to the whole statistics but only to its part. And we cannot be sure that the statistics of the sub-ensemble is the same as that of the whole ensemble.
It took over 30 years to build an experiment (more precisely, three experiments; one with electron spins and two more with photons) that really confidently refute local realism and show that quantum mechanics has to be taken seriously. One of our basic assumptions about this world thus has to be wrong; some signals are able to travel faster than light or it does not make sense to talk about objects we are not currently observing.
1 The experiment with two coins and the conclusion in this paragraph are a simplified version of the famous EPR paradox. It was formulated by Albert Einstein, Boris Podolsky, and Nathan Rosen in 1935 to show the problems of the Copenhagen interpretation of quantum mechanics.
2 Such an experiment — not with coins but with electron spins — was first proposed by John Stewart Bell in 1964. He showed that a particular correlation between the spins is bounded by the value of 2 for local realistic theories, whereas quantum mechanics allows to have stronger correlations, with values exceeding 2.