Against parachute skepticism
Or: how to read science
In the twenty-first century, research has become universally accessible. Anyone with a high school education and an internet connection can read original research.
This is a quiet modern miracle.
A critical reader is no longer forced to rely on conflicting authorities; where there’s controversy, one can turn to the actual evidence on each side. Everyone can, in principle, have an informed opinion.
In a companion article, I will walk you through how to find the most important research on a topic. Before that, though, I want to correct a common misunderstanding about how to read scientific evidence. Let’s suppose you’re trying to read the research on a question — say, ‘is sleep training harmful?’. A naïve way to read the research is:
- Find all the relevant studies.*
- Sort through them and throw out anything that looks dubious.
- Weigh up the ones that find ‘pro’ and ‘con’ evidence, and decide how strong the cases are.
I am delighted when people put in this effort; nothing annoys me more than appeals to authority. But, unfortunately, this process often gives misleading answers. The point is beautifully illustrated by a paper that concludes:
Parachute use did not reduce death or major traumatic injury when jumping from aircraft. (Yeh, 2018)
*I use ‘study’ and ‘experiment’ interchangeably below. Meanings differ subtly between fields; I’ll use both terms for ‘empirical evidence’.
That… unusual… paper is making a very serious point.
Science is not just about experimental evidence; it’s also about theory. Experiments strengthen our beliefs in theories, and theories tell us far more than experiments on their own could. If you only look at experimental evidence, you will get a misleading impression of the research. If you read to the end of this article, I’m confident I can convince you of that. You won’t need any advanced scientific or statistical knowledge; just the ability to think for yourself. I’m going to walk you through a case study, and I’ll explain all the material you need to understand it.
If you want to read more about the confusion that arises when people misunderstand how to read science, I’d recommend this excellent blog post:
For avoidance of doubt: I am taking a case study from child development, because that’s where I know the research. The case study is on circumcision, because both the theoretical background and experimental evidence are unusually compact. But these are sensitive areas…
- Child development research is not the same as popular ‘parenting advice’. In some cases, popular advice often tells people the opposite of what the research says.
- For non-US readers: I am not trying to make a point about any religion(s). A majority of male newborns are circumcised in the US.
Stop!
I need you to do something. It’ll take twenty seconds, and you need to do it now, or you won’t be convinced by this article.
The case study below looks at circumcision of newborns. Before you read on, please try to predict what long-term psychological effects circumcision might have, if any.
(This is a technique for countering the cognitive biases we all have. It’s only after you try predict something for yourself that you can get an objective sense of how impressive other people’s predictions are.)
.
.
.
.
Theory
The Force of Evolution
Can you remember your first crush? Can you remember how it colored everything? You felt that way because evolution exerts huge forces on our psychology. We have strong drives to behave in ways that make us more likely to survive and to have children.
In the environment in which we evolved, around a quarter of children died in their first year, and only half made it to puberty (Volk, 2013). This enormous evolutionary pressure means babies and children have strong drives that would have made them more likely to survive (and later procreate) in their original environment.
Of course, the world has changed. Infant mortality has dropped dramatically, and so many evolved drives aren’t needed for survival now. But our world has changed too fast; evolution will need many, many generations to ‘catch up’. So we, and our children, are stuck with the drives that fit our environment thousands of years ago.
These ideas let us make some remarkable predictions about child development. To understand them, we’ll need a few concepts from evolutionary biology…
Developmental Plasticity
An individual organism can ‘grow up’ to have different adult forms depending on what environment it grows up in. This is called developmental plasticity. For example:
- The fruit flies reflect a general biological rule that smaller forms are better suited to higher temperatures. (See Bergmann’s rule and its developmental-plasticity version, the Temperature Size Rule.)
- For the butterfly Bicyclus anynana, lower temperatures happen in the dry season, when eyespots would interfere with camouflage (Lyytinen, 2003).
A simplified but useful picture is that organisms have switches which are set in early life, and which affect what kind of adults they turn into. We’ll come back to these switches in a moment.
Life History Theory
Evolution allows more than one way to ‘do well’. Life history theory studies the evolutionary strategies that different species use. The most important kind of variation is between fast and slow strategies.
Organisms which follow fast life strategies reach sexual maturity quickly, have many offspring and invest little in each one. They also invest less in their own upkeep; they ‘live fast and die young’. Organisms which follow slow life strategies do the exact opposite. You can see the fast-slow continuum illustrated here:
These different strategies are a result of tradeoffs. Organisms have a limited supply of resources, so they can’t do everything well. If they invest more in their own upkeep, i.e. have longer lifespans, they have fewer resources for reproduction. An organism can only survive a long growth phase if there is sufficient parental investment to protect it while it is growing. And so on.
Fast and Slow Strategies in Individuals
Life history theory has also found fast-slow trade-offs for individuals in a species (Réale, 2010). So within one species:
- Some individuals follow faster strategies. They start having children earlier, have more children and invest less in them. They also live less long.
- Some individuals follow slower strategies. They start having children later, have fewer children and invest more in them. They also live longer.
Individuals tend to pick strategies that fit their environment in an evolutionary sense. Here’s an example: if you are in a dangerous environment, it doesn’t make sense to wait long to have offspring. There’s too much risk that you’ll die before having any at all. So the ‘fast’ strategy is better suited to dangerous environments.
Remember this? When individuals alter the way they develop to fit their early environment, that’s just developmental plasticity. Choosing between fast and slow life strategies is a kind of developmental plasticity; it’s just that the ‘switches’ being set affect not only physical form, but also behaviour.
For example, if an individual faces a dangerous early environment, that will set the biological ‘switches’ which make the danger-suited fast strategies more likely. These switches affect an organism’s risk tolerance, libido, and other psychological factors; those psychological effects push the organism to ‘live fast and die young’, to have children early, to have many children and to invest less in them.
Where are the Switches?
In vertebrates, the most important behaviour-affecting switches seem to be located in the hypothalamic-pituitary-adrenal axis, more often called the HPA axis. This is part of the stress response system; in humans it looks like this:
The stress response system is highly plastic: its lifelong behaviour changes in response to early experience, and particularly early stress (del Guidice, 2011; Gunnar, 2009; Levine, 2005).
Now, when I said individuals ‘choose’ strategies, I was being a little loose with language. Even in primates, it’s not a matter of conscious choice; the early environment induces biochemical changes in the stress response system and other parts of the brain. The long-term biochemistry of the brain changes in response to the early environment — the levels of different hormones change. Those hormonal changes affect risk tolerance, libido and other observable psychological factors.
Recap
What I’ve said above applies to all vertebrates, but we’re particularly interested in humans. To sum up:
- Danger in the environment causes stress, which is reflected in increased cortisol levels.
- In early life, large or sustained cortisol rises (& other similar changes) ‘flip switches’ in the stress response system.
- These flipped switches result in different lifelong levels of hormones.
- Those changes in hormone levels push an individual towards increased risk-taking, risky sexual behaviour and other parts of a faster evolutionary strategy.
This is a very simplified picture of life history theory; if you want a proper introduction, there’s an excellent one in (del Guidice, 2016).
The Nature of Danger
I’ve outlined how humans react to dangerous early experiences by switching to fast life strategies. But: what counts as dangerous?
Remember that the ‘choosing’ a life strategy isn’t conscious; it’s an automatic biological process. It’s something that we’ve evolved to do. So what matters is what was dangerous in the environment we evolved in. In many cases those things are no longer dangerous, but evolution hasn’t caught up.
If you stand just outside a cage containing a snarling lion, your adrenaline and cortisol levels will likely rise even though your conscious mind knows you are safe. What’s true for adults is doubly true for babies, whose perception of danger is almost entirely hard-wired. If something was dangerous in the evolutionary environment, babies will perceive it as dangerous, their cortisol levels will rise, and so on.
… and that’s it for the theory. Now to a case study where we can see how effective the theory is…
Neonatal Circumcision — the Experimental Evidence
Until recently, we had no experimental data on long-term psychological effects of circumcision. However: circumcision is known to cause enormous increases in the cortisol levels of newborns (Gunnar 1981).
As noted above, the stress hormone cortisol is produced by the HPA axis, where key developmental switches are located. So life history theory gave us a prediction: circumcision would push men towards ‘fast’ strategies.
Miani (2020)
The first substantial study to test these predictions was Miani (2020). I’ve bolded their key predictions about the long-term psychological effects of circumcision:
Consistent with the life-history theory, we hypothesize that adults who underwent neonatal circumcision, compared to those who did not, will display alterations in socio-affective processing conforming to the fast life-history strategies, namely, decreased reliance on social environment and increased perceived stress and sexual activity.
Specifically, here we test whether early-circumcised men, compared to men who did not undergo neonatal circumcision, display higher attachment insecurity and emotional instability (i.e., impaired functioning in emotional, motivational, and social domains), lower empathy and trust, higher sexual libido and unrestricted sociosexuality (i.e., preference of short- over long-term mating), as well as higher stress and risk taking attitudes.
We can sum up these predictions like this:
I do want to emphasise that prior to the Miani paper there was very little experimental evidence about the long-term psychological effects of early circumcision. Here’s what there was:
- Bollinger (2011). Found a link to alexithymia, or difficulty idenfitying emotions.
- Bauer (2013) and Frisch (2015). Found links to autism.
- Ullmann (2017). Looked at only 9 circumcised men (!) and found no significant results.
- Studies on sexual function, reviewed in (Morris, 2013) which concludes circumcision had no effect.
Alexithymia and autism can cause problems with empathy. But apart from that, Miani (and co-authors) had no experimental evidence supporting any of their predictions. They were made just on the basis of life history theory.
Findings
What did the paper find? This graphs sum things up:
So, for example, if you look at this part…
You can see that early-circumcised men have higher sexual drive and higher stress levels than uncircumcised men. Which is exactly what life history theory predicted. Here’s a ‘scorecard’ showing how the experimental evidence fit the other predictions:
This is a remarkable confirmation of life history theory.
Parachute Skepticism
There is no stronger proof of a theory than this: it makes new predictions, and later experiments prove those predictions correct. For example, when Einstein formulated general relativity, he used it to make two new predictions; it’s was after those predictions proved correct that people believed the theory to be correct.
If you tried to make predictions yourself, you’ll have a sense of why this is. There are so many different things that circumcision could have caused that it’s inconceivable that the predictions just happened to be right. The only plausible explanation is that life history theory encapsulates some essential truth about the world. The details may be modified and refined over time, but at heart, the theory must have captured something real.
When people miss the importance of theory, and try to look at experimental evidence in isolation… well, that’s parachute skepticism again.
Here is the crux of this article. If you look at the Miani paper in isolation, there are many, many holes you can poke in it; for example, it doesn’t correct for multiple hypothesis testing. It’s all too easy to dismiss it and conclude that we have no evidence that neonatal circumcision has long-term effects. But that conclusion utterly fails to explain how life history theory made such effective predictions. It’s like watching someone ace an exam and concluding that they guessed all the answers.
So here as in many other areas of science, the experiments on their own don’t tell you anything; it’s the combination of theory and experiment that is compelling.
In some cases, when people focus exclusively on experiments, they’re doing that for good reasons. For example, they might work on drug discovery — the human body is so complex and so messy that our theories of it are very poor, so we are forced to rely mostly on raw experimental evidence.
More often, there’s an insidious phenomenon at play. As Kraft (2015), a study of motivated reasoning and scientific beliefs, notes:
Partisans … systematically denigrate, depreciate and counterargue evidence that is contrary to their political views but accept uncritically the supportive evidence.
In the social sciences, you can reject any research in a seemingly scientific manner. Saying ‘causation is not correlation’, talking about the replication crisis, pointing out possible confounders and, of course, parachute skepticism will let you disregard any paper.
You can imagine a perfect philosopher-scientist who does this systematically and lives with a deep sense of doubt about the world. But that’s not what happens in practice. As Kraft (2015) and much other research have found, people systematically use those superficially scientific techniques against just the findings that make them uncomfortable.
I’ll have more to say about this in a sequel article. For now, I’ll just plead: when you see people use these skeptical techniques, ask yourself if they’re applying them in a balanced way. Are they being skeptical about findings they are threatened by and those they are comfortable with? For example, in child development, are they being equally skeptical about things found to be damaging to children and things found to be beneficial? Or do they just denigrate, depreciate and counterargue the findings they feel threatened by? If so, that’s classic science denial.
Which theories work?
This article has been about how you should not read science. I’d like to close by telling you more about how you should read it. Unfortunately, that’s just hard. I’ve written about the importance of theories, but not all theories in social science are backed by solid evidence. There’s a world of difference between
- theories that only ‘explain’ pre-existing data
- theories that have made new predictions, later proven correct
The difference between these is like the difference between taking an exam paper you’ve seen beforehand and one you haven’t! But if you’re coming to a field from scratch, it’s difficult to get a sense of which theories fall into which category. You have to work through the material yourself to be really convinced.
I do strongly feel that whenever an expert makes an assertion, they should sketch the evidence — both theoretical and experimental — that supports their position. A roadmap, if you like, so you know where to start reading. I’d encourage you to hold everyone to that standard; it helps keep people honest!
Afterword: Reading about child development…
I know some are readers are here because you’re interested in how children develop, so let me close by giving you some pointers to further reading.
I wrote about life history theory and circumcision because they can be discussed compactly. But the most powerful theory, the one that has proven to have the most explanatory power, is attachment theory (Bowlby, 1958; Ainsworth, 1978).
Before I go any further: attachment theory is not the same as attachment parenting! Attachment parenting was originally called immersion parenting; William and Martha Sears changed its name to give it a veneer of scientific credibility.
The most detailed experimental work confirming the predictions of attachment theory is covered in The Development of the Person (Sroufe, 2005). It’s a remarkable book, and if you want to understand more about how children develop, I’d recommend it in the strongest terms.
Beyond that, watch out for easy answers; anything that suggests ‘one simple trick’ will improve children’s wellbeing is likely preying on parents. Watch out for behavioural findings that don’t have an explanation; as in the example above, solid work is generally backed by evolutionary explanations and grounded in biological and neurological evidence. And finally, watch out for political bias. Child development, sadly, has to deal with more than its share of politics, and that can mean that headline findings are misleading.
…or not reading about child development!
In case you’re a parent, and that seems overwhelming, let me say this. You don’t need to read deeply into child development to be a good parent. Just as children have evolved to survive, you’ve evolved to help them survive. Babies will let you know when something seems threatening to them!* If you follow your instincts and soothe them, things are likely to turn out fine.
*NB. This is not true if you keep ignoring babies when they cry. Crying uses a lot of energy (Barr, 1998), so babies stop crying if parents don’t respond — but they remain just as stressed as before (Middlemiss, 2012).
So my advice to parents would generally be, relax, do what comes naturally, and don’t worry about keeping up with the latest research. Your time is much better spent having fun with your children!
Thank you for reading.
