Prediction is very difficult, especially if it’s about the future—Niels Bohr

Lisa Randall is a professor of theoretical physics at Harvard. Her research has touched on many of the basic questions of modern physics: supersymmetry, Standard Model observables, cosmological inflation, baryogenesis, grand unified theories, and general relativity. She has also written popular books about her work and science in general. Thus she has a handle on aspects of science that overlap my expertise—not to mention those of her sister Dana Randall, whom I have known as a colleague for many years.

Today, Ken and I thought we would talk about recent developments in particle physics, and their connection to two topics dear to us.

Randall’s most recent popular book is Dark Matter and the Dinosaurs. The idea she advances is that the periodic extinctions in Earth’s history may have been caused when the solar system passes through a plane of dark matter within our galaxy. But dark matter and also dark energy have come under increasing recent doubt, even from their original formulator. Maybe Niels Bohr’s quote should also say:

Prediction is very difficult, especially if it’s about the past.

Randall’s previous book, Knocking on Heaven’s Door, is most relevant to this post. The 1973 Bob Dylan song title it pinches describes the feeling of doing frontier physical science. Insofar as her own work is mostly theoretical, much of it connects to feelings we have in computer science—especially complexity lower bounds where the door mostly feels slammed shut.

But the book is also about the practice of experimental science—not only how to gather knowledge but when and how we can have confidence in it. Its long middle part is titled, “Machines, Measurements, and Probability.” All three elements are foremost in considering a new development that involves two measurements taken 20 years apart.

Muons

Last Tuesday’s New York Times highlighted a potential discovery in particle physics. It was in their Tuesday science section.

The result is an experimental discovery that could show that the current model of matter is wrong.

“This is our Mars rover landing moment,” said Chris Polly, a physicist at the Fermi National Accelerator Laboratory, or Fermilab, in Batavia, Ill., who has been working toward this finding for most of his career.

Indeed. It is a Mars landing moment. They both involved many people, lots of exotic machinery, lots of money, many years. Say three billion dollars or so for Mars. Say nearly the same amount for muons—the annual budget for Fermilab is over one half billion dollars. It was certainly enough to reassemble and upgrade a huge accelerator ring that was first used at Brookhaven National Lab on Long Island in 2001:

NY Times src

The study used the muon to probe the Standard Model of physics. Muons are useful because they are charged like an electron, which helps control them in an accelerator. Yet their mass is roughly 207 times larger than an electron. The same charge helps control their motion and the large mass makes collisions more interesting. As Polly stated in Natalie Wolchover’s story for Quanta:

“[I]f you’re looking for particles that could explain the missing mass of the universe—dark matter—or you’re looking for [supersymmetry], that’s where the muon has a unique role.”

Theory and Jealousy

Computer science theory is so different from high end physics. We are closer to the type of research that Randall does. We involve few people, no exotic machinery, and small amounts of money. Maybe the closest attribute we have to high-end research is we also take years and years.

Perhaps we are also jealous of high-end physics. Not just for money, but for the ability of particle physicists to get announcements into the New York Times. Polly said the following about the ending day of the muon experiment two decades ago:

When we revealed the results, people from all over the world flew in to visit the lab. These experiments take decades to build and analyze, so you don’t get to go to very many of these events. We did a little “Drumroll, please” and then had the postdoc managing the spreadsheet hit the button to show it on the projector. Lo and behold, you could see that there was still a three-sigma discrepancy!

At the time he was a graduate assistant assigned to machinery for measuring particle energies. He had fixed a problem where someone had touched a component with bare hands and thereby ruined its sheathing. All such problems were meant to be ironed-out by the drum-roll event. But all this raises two further interesting issues that connect the muon results with issues we think about in computer science. Let’s look at them next.

Three to Four Sigma

In an experimental science one must be aware that results are not exact. They are samples from some random process. Flip a coin 10 times in a row. If they all come up heads what does that mean? Could be the coin is fair but this happens about one time in a thousand. Or the coin is biased. Or something else.

Flip a muon many times. That is sample some muon experiment. The outcome is from a random process. Some of it comes from properties of the natural processes themselves and others from incidentals of the measurement apparatus. How do we decide if the experiment means what we think it does?

The theory developed by Carl Gauss and others before and after to delineate the normal distribution was largely prompted by analysis of measurement errors to begin with. This yields the “rule of three,” about the percentage of values that naturally lie within an interval estimate in a normal distribution: 68%, 95%, and 99.7% of the values lie within one, two, and three standard deviations of the mean, respectively.

The question is, how to assess cases where the measurement result is well outside these intervals—when can we conclude it is more than a deviation by natural chance? In social sciences a result is “significant” provided it lies outside two-sigma. In particle physics, there is a convention of a five-sigma effect (99.99994% confidence) being required to qualify as a discovery. No Nobel prize for less.

The situation with the muons has an extra factor of repeated measurements—but there have been only two measurements so far:

Composite of src1, src2

The blue line in the left figure is the original Brookhaven measurement; the red is the new one. There is also theoretical uncertainty in the calculation of the Standard Model prediction, and that combines with the measurement error bars to give the sigma baseline. The chart at right normalizes the deviation to parts per billion—the measurements need to be incredibly fine. This scale appears to be about 25% under the current -scale (it shows about for Brookhaven compared to its after revised uncertainty) but it is close enough to get the picture.

Although the new Fermilab result by itself deviates slightly less from the Standard Model, it corroborates the earlier measurement. It is not fully independent from it, but the combination is enough to raise the current claimed deviation to about sigmas. This is well above social science level but below Nobel level. This is with respect to the probability that the effects are real.

The social significance of is that it is above the “” level where hoped-for anomalies have subsequently disappeared for reasons chalked up to natural chance. This is because physicists around the world do many a hundredfold amount of hopeful measurements. Some measurements get initial “bumps” up just because of the numbers. But reduces the natural frequency under 1-in-40,000. This is why reproducing measurements is so important, why the new Fermilab team devoted all the expense and effort. A more independent measurement on other machines could give a higher boost that might get over the line. Time to break out the wallets and hammers?

The is not, however, beyond the realm of recent experience with apparatus faults and modeling error. On the latter, there is still some doubt about the theoretical prediction for the muon’s magnetic moment. In any event, the muon results are exciting but still below what is required for a true discovery. Time will tell.

The second factor we draw attention to concerns the human “hoping” directly.

Blinding

In any experimental science one must also be aware that people are not unbiased. Scientists have much invested in the outcome of their experiments. Think jobs, tenure even, funding, and more. So a big physics experiment like the muon one must be careful. They follow standard practice to perform blinded data analysis.

This surprised me. This blinding is a crypto-type protocol, which is something we computer scientists study. The muon team performed a protocol that protected against cheating. Here is how they did it:

In this case, the master clock that keeps track of the muons’ wobble had been set to a rate unknown to the researchers. The figure was sealed in envelopes that were locked in the offices at Fermilab and the University of Washington in Seattle.

In a ceremony on Feb. 25 that was recorded on video and watched around the world on Zoom, Dr. Polly opened the Fermilab envelope and David Hertzog from the University of Washington opened the Seattle envelope. The number inside was entered into a spreadsheet, providing a key to all the data, and the result popped out to a chorus of wows.

“That really led to a really exciting moment, because nobody on the collaboration knew the answer until the same moment,” said Saskia Charity, a Fermilab postdoctoral fellow who has been working remotely from Liverpool, England, during the pandemic.

This mechanism for blinding suggests possible crypto questions. They hid the master clock rate. Can this be modeled as one of our crypto problems? Can we prove some security bounds? If they claim that hiding the rate protects against cheating then they should be able to make this claim precise. The discovery of gravitational waves used a blind injection scheme tailored for that experiment. How can this be generalized?

Open Problems

We have discussed two aspects that involve soft numbers rather than hard machines and hard-shelled particles. Perhaps they are interesting new problems for us? What do you think?