Rationally ignorant physicists and the Theoretical Minimum

Recently, i’ve decided to table my computational research project and spend the rest of the summer reading some graduate-level physics. The challenge I’ve picked for myself was the notorious ten-volume Course of Theoretical Physics by Lev Landau and his students; the series is commonly known among many physicists as the gold standard reference in theoretical physics. The books (let’s call them Landafshitz like many physicists affectionately do) are not really known for being pedagogical; in fact, the Landafshitz is concise enough that one “trivial” or “obvious” statement (words from the books themselves) might require three pages of carefully written mathematical proof. Thus far it has been clear to me that I will not be able to finish the set, not even the first volume (non-relativistic classical mechanics), within this summer. I’ve decided that it would be an appropriate thing to read for the entire length of graduate school, which speaks volumes (pun not intended) about the sheer breadth and depth of the series.

In political science and economics, there is a concept called “rational ignorance”, which concerns social choice and the degree to which people learn about the consequences of their decisions. Let’s consider this: in a democratic election, who would you vote for? In an ideal world we say that each member of society votes for one who they consider to represent their best interests. That being said, it is not always, if never, obvious which candidate represents one’s such best interests. The process of learning about candidates incur some opportunity cost on the individual voters. Balancing between the cost of being politically informed and the modest benefit of one single vote for the right candidate results in what we call “rational ignorance”: voters generally know enough to make a decision, but they do not know too much that it takes over resources allocated to other tasks of life.

Let’s go back to the good old Lev Landau. Himself a renowned theoretical physicist of the 20th century, Landau developed a comprehensive examination of physics to offer to those bleary-eyed graduate students who desire to come under his scholarly patronage: the “Theoretical Minimum”. Despite the name, only 43 people passed over several decades the exam was offered. Only ones who passed would be considered worthy of Landau’s mentorship and sufficiently prepared for a productive career in theoretical physics. Course of Theoretical Physics was eventually developed by Landau as a set of textbooks and study guides to the exam.

So what does rational ignorance have to do with physics? If you agree with Landau (and many others who agree with him), you’ll see that it takes years of studying all different fields of physics, covering the entire Landafshitz, to be able to touch the fringes of theoretical physics and push it an inch further. That learning does not come without a cost; theoretical physicists as mortals only have a limited productive amount of time that they might choose to spend on learning, researching, teaching, or other scholarly pursuits. Of course, one cannot fully decouple learning from researching as the productive scientist must constantly engage with what their colleagues are working on, but they are still quite different tasks regardless. The cost is somewhat smaller in experimental sciences: you might not know all the theory right now, but that does not preclude you from doing the experiments with your hands and your instruments. Theory perhaps has a higher stake, since how do you even do theory if you do not even understand what the frontiers in the field are doing? For theorists, the acts of thinking are one with the acts of doing; for experimentalists they are co-dependent but separate. The more physics we have, the higher the Theoretical Minimum is for theorists. In fact, since the time of Landau, we’ve discovered density functional theory, quantum chromodynamics and quantum computing, among others, and more physics is still being produced every year. It is unlikely to be a coincidence that since Landau’s death in 1968, there have not been physicists that contributed so substantially to all fields of physics—a six-decade dry spell. Perhaps the Theoretical Minimum was now too high the trade-off between learning and doing forces theorists of our time to hyperspecialize? Admittedly, I do not have a good answer for this without further digging for evidence.

A theoretical dysphoria is what I use to call the sense when you are fascinated with the general prospects of working as a theorist but not learned enough to even come up with an executable idea. In other words, you watched too much Oppenheimer and knew too much physics to know exactly what is going on, but at the same time you don’t know enough physics to actually work like Oppenheimer himself. I am sure mathematicians have it too: can you really become the next Perelman or Terry Tao if you have not yet recovered from the trauma inflicted by elementary real analysis? And I have it too, shamelessly admitting. Of course, there is no doubt there is no replacement for learning if I ever want to make a substantial contribution to theory of any sort, but the idea that we found so much more physics that the way our society does theoretical physics 100 years ago is no longer sustainable bothers me. Can we have another Paul Dirac who built the entire quantum mechanics in his doctoral dissertation? I do not see how that would work in the modern day.

At the end, the theorists of our time must choose the amount of ignorance rational for them; the non-scholarly forces of “publish or perish,” tenure, and a towering sunk cost push for more, while the interests of the scientific enterprise and the human society as a whole call for less.