If you’re going into surgery, you want the youngest operating surgeon available.

This is a slight exaggeration – you don’t want a doctor in their first year out of medical school.[1] After that, it’s less clear. One review found thirty-two studies indicating that the older a doctor was, the worse their medical outcomes; that review only found one study indicating that all outcomes got better with increasing age.[2] Other analyses suggest that middle-aged doctors might do better than younger doctors (though the effect is not statistically significant)[3], but older doctors are still clearly worse than middle-aged doctors.[4]

It’s not like doctors become terrible with more experience, but they are measurably worse. In one study, an extra twenty years of experience translated to about one additional elderly patient dying out of every hundred treated.[5] Why would twenty extra years of practice make a doctor worse at helping people?

Similarly, some research on famous painters, writers[6], and composers[7] found that it’s most common for these artists to produce their best work before the age of 45. A famous composer is more likely to have written their best piece of music in their 20s than their 40s, and they’re almost twice as likely to have written their best piece in their 30s instead of their 50s.[8]

Again, creatives are doing worse after a couple of extra decades of experience. This phenomenon is called skill plateaus. The idea is that performance stops improving after a relatively short time. After that, putting in thousands more hours on the job won’t reliably make you much better.

What’s going on here?

Deliberate practice                      

Anders Ericsson thought he had the answer. Ericsson was a psychologist famous for his research on expertise and human performance. He claims that most people practice a new skill until they can perform it adequately for their purposes.

For example, someone might practice tennis until they’re good enough to play with their friends. At that point, they stop practicing to just enjoy the casual games. Many people assume that they’ll continue to slowly get better just by playing games. However, that doesn’t seem to be the case. If they just play games using the skills they already know, they stagnate or even get worse over time.[9]

Ericsson sought to study what separated the truly outstanding top performers from those who were merely good. He concluded that the key factor was a particular approach to improvement, which he called “deliberate practice”.

Deliberate practice is not just any focused practice.

Deliberate practice is breaking your work down into individual steps, finding the optimal technique for each step, and practicing each technique until you execute it automatically. It requires clear, rapid feedback so that you know what you’re doing right and what you need to work on (this feedback usually comes from a coach).

Say our amateur tennis player wanted to continue improving. They might get a tennis lesson. The coach would identify what is holding the player back -- perhaps they hold the racket poorly. Here the coach breaks tennis down into individual subskills and identifies which are making their performance worse.

So, the coach demonstrates a better way to grip the racket and marks their racket at the correct place to grip. Here the coach gives the player a better technique and a training method to practice it.

The player practices gripping the racket correctly, takes one swing, and then stops to check their grip. Here the player repeatedly practices the skill so that they learn it in their muscle memory. They get feedback by checking whether their hands align with the coach’s marks on the racket after each swing. They might get more feedback if the coach corrects their grip as they practice.

After repeating the drill a few dozen times, they have mastered the better grip and are ready to move on to the next improvement.

These repeated small loops are good for improving your skill effectively, but don’t try them during a game! During the practice loops, you practice one tiny skill over and over until it’s automatic. Then during a real game, you put all of the tiny skill improvements together to play better. Experts have often practiced hundreds or thousands of these individual subskills to build up their expertise.

In Ericsson’s mind, ceasing to deliberately practice causes the performance plateau. People stop learning better techniques. Their performance slowly worsens over time as their autopilot skills atrophy.

Does his theory hold up?

Does deliberate practice explain which people are successful?

There’s a good amount of evidence that deliberate practice can work well – and a lot of caveats.

There are plenty of studies supporting deliberate practice. However, I focused on the studies trying to disprove deliberate practice to see what the counter arguments were.

The result was kind of funny. The authors of one such paper wrote a New York Times piece called “Sorry, Strivers: Talent Matters” attempting to debunk deliberate practice by highlighting their finding that working memory determined 7% of success in their study. Meanwhile, their study also found that deliberate practice determined 45% of success.[10]

Another study found that deliberate practice “only” determined one third of success[11], and a meta-analysis found that deliberate practice only predicted 11%.[12] Even the naysayers agree that deliberate practice is important; they just don’t believe that it entirely explains success.

The skeptics could have made a stronger argument by pointing at the limited domains in which deliberate practice is strongest.

In the above studies, the ones finding that deliberate practice accounts for 45% and one third of variance in performance were only looking at highly developed fields like music and chess. Highly developed fields have objective methods of evaluating performance (such as music competitions or chess ratings) and effective training methods, usually developed over decades or centuries. Meanwhile, the meta-analysis (which found that deliberate practice explained 11% of the variance in performance) looked at multiple fields.

In highly developed fields like music and chess, this meta-analysis also found that deliberate practice was important, predicting more than 20% of success. Yet, it found that deliberate practice explained only 1% of the variance for professional performances such as computer coding or selling insurance.

This feels a bit like the growth mindset debate. “Researcher becomes famous by discovering the secret of learning. Critics later claim that the effect is bogus. Fierce back and forth ensues.” Likely the result will be similar here: “Yes, the effect is probably real, just smaller than everyone initially assumed when reading glowing news articles.”

But why this discrepancy?

Why would deliberate practice work so well when learning chess, and so poorly in professional work?

Remember that deliberate practice is about finding better techniques.

If you have to invent these good techniques from scratch, this is really hard. If other people have put thousands of hours into identifying the best techniques for doing a task, and the best training methods for those techniques, then it is easy.

In one of the early deliberate practice studies, the study subject started out able to remember a string of seven numbers. Once he developed a system for grouping numbers, he could remember up to 18. But then he struggled to continue learning.

He had to develop a meta-system for remembering groups of numbers before he could break that plateau. It took him more than 200 hours of practice to develop the techniques to be able to remember nearly 80 digits.[13] That’s a lot of time to develop techniques from scratch for a relatively simple task.

Ericsson claims that running, swimming, piano, and countless other fields have far higher standards today than they did a hundred years ago – because we’ve learned better techniques and training methods.[14] It took Newton to invent calculus, but since then we’ve refined the field and discovered better teaching methods. Now it's a skill that can be learned by the average high school student.

Meanwhile, the meta-analysis lumped computer programming, military aircraft piloting, soccer refereeing, and insurance selling under “professional performances.” None of these fields have refined their optimal techniques or training methods as much as chess or piano has.

It seems possible that doctors and creatives stopped practicing because they couldn’t find new techniques to practice. Maybe we occasionally get a good, easy-to-use technique that lots of doctors adopt, like using checklists that reduce patient mortality by 23%.

But maybe most of the time, doctors aren’t sure what sub-skills would reduce mortality or improve their bedside manner. Furthermore, taking several hours each week to practice -- instead of producing immediate output -- is hard. So maybe the doctors don’t practice anything if they’re not certain what would make them better.

It’s not clear which elements of deliberate practice are essential for skill improvement. Doctors could find some bottlenecks, look up techniques, and practice it themselves. How would the results compare to deliberate practice? I’m not sure yet.

However, even in the contexts like professional sports and music -- where people are highly incentivized to deliberately practice and coached on well-refined techniques – deliberate practice explains less than half of the variance in success. That should be our upper bound on how useful deliberate practice is.

What else explains why doctors don’t get better?

I think the people who say  “Talent matters!” are pointing at a big chunk of the answer.[15] Fortunately, we can build a more detailed model of what it means to say someone is “talented”.

Psychiatrist Scott Alexander looked at med school data and found that medical students’ scores plateaued after about their third year. Students got steadily better in their first year. In year two, they learned a bit less but were still improving. But after year three, they barely increased their test scores. “Compare fourth-year and fifth-year surgeons, and it’s pretty close to 50-50 which of them will know more surgery.”

Scott says students in his program were still spending just as much time attempting to learn in their fourth and fifth years. They were far below the maximum score, and some students were doing better than other students. So why would their learning level off?

Because human minds can’t remember the entirety of medical knowledge.

A cognitive constraint is a limitation to how much we can learn that is imposed by the architecture and processing capabilities of the human brain.

One deliberate practice study found that working memory explained 7% of success at sight reading music. Working memory capacity is one cognitive constraint at the point of learning. Other constraints impact the ability to remember information over time or process new information efficiently.

These cognitive constraints differ from person to person, and can explain what we mean when we talk about “talent”. If someone is less limited by the constraints that bottleneck other people, they improve more quickly and seem naturally talented.

For example, people forget things over time unless they review the information at some gradually expanding intervals. (This is the widely accepted idea behind spaced repetition learning techniques.) Scott suggests that in order to remember a fact, people need to review that fact within different time intervals: some might need to review it after a week, others after a few months. This means that some people build up larger memory trees than others.[16] 

Scott writes: “Suppose that you forget any fact you haven’t reviewed in X amount of time (X might be shorter or longer depending on your intelligence/memory/talent). And suppose that an average doctor sees 5 diseases ~weekly, another 5 diseases ~monthly, and another 5 diseases ~yearly. A bad doctor might forget anything she sees less than once a week, a mediocre doctor might forget anything she sees less than once a month, and a great doctor might forget anything she sees less than once a year. So, the bad doctor will end up knowing about 5 diseases, the mediocre doctor 10, and the great doctor 15.”

I’m guessing these constraints explain a lot of the rest of skill plateaus. Individuals have a certain range of ability to improve. For example, doctors have some limit to how many medical facts they can memorize. If medical students are already hitting their limit after three years, no wonder doctors don’t keep improving for decades.

Forget the old myth that humans only use 10% of their brains. If doctors are constantly operating at their max cognitive capacity, they’re already at 100%. If this is the case, we should expect any small cognitive decline as they age to translate directly into decreased performance. If the doctor was operating at 100% of their ability and then that ability goes down by 1%, we should expect them to only operate at 99% of their former capacity.

Can we tease apart whether cognitive constraints or lack of deliberate practice contributes more to skill plateaus?

To attempt to disentangle which factors are contributing to skill plateaus, we can check if a skill known for deliberate practice follows the same skill plateau pattern as doctors and creatives. Doctors and creatives possibly get better until their thirties or early forties, then they get worse.[17]

If deliberate practice can solve skill plateaus, we should expect to see chess players continue to improve their skills later in life. They have more skills to keep practicing, so they can continue to improve over time.

However, if cognitive constraints are the main factor, we would expect to see chess players following the same pattern of decline as doctors and creatives.

Do chess players also get steadily worse after their mid 30s?

Mostly yes, but there’s a twist.

Studies on chess players found that their performance declined after the age of 35 or 40.[18] Younger players got sharply better, then their performance tapered off as they got older. Additionally, if you look at the top chess players, almost all are in their twenties and thirties.[19] So, chess players fit a similar pattern of decline to doctors and creatives.

However, chess players also demonstrated a cohort shift.

Over a hundred years, players got better. Subsequent generations made a higher percentage of optimal moves than the previous generation. This pattern has persisted across four generations.

Additionally, younger players experienced faster improvement earlier in life compared to previous generations.

My guess is that there are two things going on here.

First, across a field, practitioners on average decline after approximately the age of 40. Doctors, creatives, and chess players all seem to get worse at their craft after about that age.

This could be consistent with a story where “just doing the job” actually improves performance for the first few decades; and with a different story where they do deliberate practice but run out of obvious techniques after 20 years.

However, I’m guessing this pattern is better explained by cognitive decline. We found similar results in each field regardless of whether the field tends to do deliberate practice, which makes me feel that this pattern is less likely explainable by deliberate practice.

Similarly, if raw talent just improves skills for a while, that doesn’t explain why that pattern would reverse and people start getting worse. The theory of deliberate practice tried to explain this by saying that skills decay without deliberate practice, but the “just doing the job” case doesn’t have that excuse.

If people are already hitting their cognitive constraints given the current techniques, then performance will go down with even slight cognitive decline. If that decline is fairly stable across age, then we’d expect to see a pattern like this. (Note: all of these examples indicated fairly gradual performance declines, so we probably don’t need to panic until at least our sixties.)

Second, within a field, practice probably explains part of success, though it might depend a lot on the field. Deliberate practice probably doesn’t explain why people start getting worse in their 40s, but it might explain which already-talented people become the best.

The original deliberate practice research focused on what separated the very best from the merely decent within a highly specialized field. If you compare two highly skilled practitioners within a highly developed field, it seems plausible that deliberate practice determines up to half of which is better.

The finding that the learning curve for chess was shifting earlier and earlier introduces another possibility. Perhaps regardless of where the max performance tops out, deliberate practice speeds up the learning curve. A chess player or doctor will learn better skills more quickly, so they have more time to use their skills.

What do we actually do with this?

I set out to investigate whether deliberate practice or cognitive constraints could better explain skill plateaus. Now, I’m guessing the answer is mostly cognitive constraints, at least for the original definition of skill plateaus. 

However, if I ask “which is more important to focus on?”, then deliberate practice matters more. We can’t yet prevent cognitive decline, but we can do something about deliberate practice.

When done well, deliberate practice – i.e. better techniques and teaching methods – makes a big difference in continued improvement. For the medical students, some practice did matter. Students’ scores definitely went up for the first few years, and I doubt Frank Abagnale[20] learned as much pretending to be a doctor as med students do in school.

Furthermore, cognitive constraints and deliberate practice aren’t totally independent.

My guess is that cognitive limitations definitely matter, but they are more like a multiplier for the techniques you’re using. Without deliberate practice, everyone just does whatever random technique they stumbled upon when they first learned to do the thing. If the common techniques are cognitively demanding, then cognitive constraints will stop most people from becoming really good at the skill.

However, if you give them less cognitively demanding frameworks or tools, they will keep getting better. So we should be able to mitigate cognitive constraints with better techniques or better training methods.

So, as long as the natural ability to memorize symptoms determines doctor quality, then doctors will be limited by their cognitive constraints to a certain range of skill. If we start having doctors do spaced repetition or give them an AI tool that reminds the doctor of unusual options, that limit changes.

This would imply that we can raise the bar for entire fields, but we need specific skills to practice.

Maybe individuals should put more effort into deliberate practice. More importantly, societies need people researching and developing better techniques and training methods so that individuals can do deliberate practice.[21]

Footnotes:

Thanks to Amber Ace for editing.