Ben Willenbring

View Original

Illusions of Explanatory Depth (IOED)

Benjamin Willenbring

Exploded view of crossbow

example: tell me what a house is

Imagine an alien comes to Earth, and asks you to explain the concept of houses. You agree, and start in – eagerly at first – but slowly realize all kinds of gaps in your knowledge.

  • At what point in history did humans begin constructing houses?

  • What are the different kinds of houses?

  • How are houses built, and what are the common building materials?

  • How are the prices of houses determined, and what are the laws governing their construction, modification, and sale?

You realize that you actually know much less than you thought.

IOED + Yale Grad Students + Toilets

Leon Rozenblit and Frank Keil are a pair of psychologists who asked Yale graduate students to explain, in as much detail as possible, their understanding of common devices (eg: toilets), procedures (eg: how to tie a bow tie), and natural phenomena (eg: how earthquakes occur). The full list is hilarious, and I’ve put it at the bottom of this page.

The instructions

What happened In the Study?

Everyone overestimated their knowledge. You can read about it in this wonderful paper: The misunderstood limits of folk science: an illusion of explanatory depth.

IOED + Software Engineering

Software engineers are just as delusional as Yale grad students. When you’re part of an engineering team, it’s normal to become superficially familiar with a TON of concepts, and then feel like you know a lot more than you do (even if you don’t write code 🤣). If you’re a full-stack developer, you’ve likely heard of some or all of these concepts and tools:

  • Prototypal vs Class inheritance, ()=> and the binding of this

  • Concepts like tech debt, cylomatic complexity, and big O notation

  • Syntactic differences between TypeScript vs. JavaScript vs JavaScript ES5, ES6, ES7, ES8, ES9, ES10

  • The concept of transpiling, the use of polyfills and shims, and tools like Babel, Webpack, SASS, Grunt, Gulp, ESLint, and Prettier

  • CI/CD, the difference between CI and CD, the use of green/blue deployments, A/B testing, and feature flags

  • Containerization, Docker, Docker-Compose, Kubernetes, pods, clusters, and service meshes

  • Serverless applications, infrastructure as code, and ephemeral services

☝🏽 That is a lot of stuff, and it’s not even 20% of the surface area of what a typical engineering team talks about in a single sprint.

Making software in 2021 is complicated

Back in 1993, the pool boy could build a web site. That’s no longer the case. The task of building and releasing Software has become so complex and differentiated, one person cannot possibly know a lot about a lot. At best, you can know a lot about a low number of things, and have superficial knowledge of lots of adjacent topics. But in general, there are too many sub-disciplines within software engineering for a single person to know how to design, build, test, and deploy an application from scratch… without receiving massive amounts of help in the form of:

  • A tricked out code editor

  • Self-scaffolding frameworks with boilerplate code

  • CI and/or some kind of build server

  • Containerized services for testing, auth, db, file storage, notifications, queueing, etc.

  • Desktop and cloud-based automation tools

  • Github gists + Stack Overflow + a shit-load of Google searches

  • More 😢

It’s natural for people who work in software engineering to feel overwhelmed. One coping mechanism is to pretend you understand more than you do, rather than saying “I don’t know”. I don’t recommend this – I’ve tried it.

Reference Borrowing (to the rescue?)

There is a thing called reference borrowing – shorthand for Saul Kripke’s causal theory of reference – which basically says that when we use names and tokens in language, there’s no burden of proof that has to be surmounted. So we can feel comfortable saying things like…

  • Plato was the teacher of Aristotle

  • Mark Twain was a master storyteller

  • Einstein developed general and specific theories of relativity

… without having met any of the people above, or possessing a paper trail of documentation to underwrite our claims. We can meaningfully make all those statements without tying philosophers into knots – which happens more than you’d think. According to Kripke’s theory, when we use names like Plato, we’re participating in a causal chain of linguistic references that ultimately lead back to an initial event, at which a name and its meaning were bound by an original utterer. In essence, there exists some chain of events that connects our utterance of Plato was the teacher of Aristotle all the way back to the flesh and blood Plato himself, along with some person’s observation that, gee there goes the teacher of Aristotle.

Modern Cognitive Science and Psychology

Philip Fernbach and Steven Sloman have modernized and simplified Kripke’s ideas, and framed them in ways that I think resonate well with any internet-literate audience.

The Evolutionary Biology Spin on Reference Borrowing

Steven Pinker has observed that one of humans’ greatest talents is the ability to “shape events in each other's brains with exquisite precision.” I’d say it’s the killer feature.

Also…

Language facilitates large-scale cooperation, which in turn alters the brain

Joseph Henrich has written an astounding book, tracking how specific cultural events like the Protestant Reformation altered our brain development.

Use it or lose it

The brains of London cab drivers are different in 2021 than they were in 1991 because the availability of cheap GPS technology has de-emphasized the need to develop enlarged hippocampuses.

Reference Borrowing is a feature, not a bug

The world of 2021 is orders of magnitude more complex than the world of 1981. We’re living in an age where we have no idea what set of skills and knowledge will be valuable 30 years from today.

When I think of the stuff my daughters have to know, just to get through their school day, I’m grateful that they don’t have to waste a single calorie of energy on unnecessary tasks like remembering to rewind vcr tapes before they return them to the store.

The Secret Sauce: Collective Fictions

Yuval Noah Harari says that humans are very good at efficiently deploying a wide array of collective fictions – eg: religion, money, laws, human rights, systems of credit, Facebook likes – that are completely imaginary, yet serve a social purpose in the furtherance of civilization’s “agenda”. Kind of like a gigantic game of make-believe that everyone plays, either knowingly or in blissful ignorance. I believe that reference borrowing is one of the premier tools in our repertoire that allows every man, woman, and child to get in on the action.

Reference Borrowing !== IOED

Reference borrowing is a facilitating tactic that allows language participants to engage in a wide array of discourses, confidently saying things like…

  • North Korea is a rogue state

  • Quantum computing is rad

  • The multiverse is real

… without having to earn multiple phd’s. And without making a mountain of preliminary clarifications, justifications, and caveats. People can just talk and say things, and hopefully good ideas will sprout forth. But this occurs with the presumption of objective factual accuracy: that at the endpoint of each speaker’s chain of causal references, lies a true proposition which could theoretically be validated by some mechanical procedure. Academic folks just call this intellectual honesty.

IOED does not do the same thing. It insists that it’s right, and is resistant to the introduction of inconvenient conflicting truths – an approach popularized by Kellyanne Conway and her alternative facts “school of thought”.

The Dunning Kruger Effect

In 1995, McArthur Wheeler walked into two Pittsburgh banks and robbed them in broad daylight, with no visible attempt at disguise. He was arrested later that night, less than an hour after videotapes of him taken from surveil-lance cameras were broadcast on the 11 o'clock news. When police later showed him the surveillance tapes, Mr. Wheeler stared in incredulity. “But I wore the juice,” he mumbled. Apparently, Mr. Wheeler was under the impression that rubbing one's face with lemon juice rendered it invisible to videotape cameras.

Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments

Self-assessments of competence, when performed by the incompetent, are grossly inflated. The reverse is true of highly competent people: they under-estimate their knowledge.

Can you identify the person responsible for these self-assessments of competence?

  • “I understand the tax laws better than almost anyone.”

  • “I know more about renewables than any human being on Earth.”

  • “I understand the power of Facebook maybe better than almost anybody.”

  • “Nobody knows more about debt. I'm like the king.”

  • “Nobody knows banking better than I do.”

  • “I understand money better than anybody.”

  • “Nobody knows more about trade than me.”

  • “Nobody in the history of this country has ever known so much about infrastructure as [me].”

  • “There's nobody bigger or better at the military than I am.”

  • “I know more about ISIS [the Islamic State militant group] than the generals do.”

  • “I know more about drones than anybody.”

  • “Nobody knows more about technology than me.”

  • “Nobody in the history of this country has ever known so much about infrastructure as [me].”

  • “I know a lot about wind.”

  • “I know more about renewables than any human being on Earth.”

  • “I know more about courts than any human being on Earth.”

See this content in the original post

Full List of IOED devices in the Rozenblit/Keil study

Below is the list of common devices that Rozenblit and Keil asked Yale graduate students to descibe in detail:

  1. How a sewing machine works

  2. How a flush toilet operates

  3. How an LCD screen works

  4. How a hydroelectric turbine changes water pressure into electricity

  5. How a can opener works

  6. How a car battery stores electricity

  7. How a 35 mm camera (single-lens reflex camera) makes images on film

  8. How a jet engine produces thrust

  9. How a zipper works

  10. How a self-winding watch runs without batteries

  11. How a cellular phone works

  12. How a microchip processes information

  13. How a greenhouse works

  14. How the U.S. Supreme Court determines the constitutionality of laws

  15. How a fluorescent light works

  16. How a photocopier makes copies

  17. How a nuclear reactor produces electricity

  18. How a car ignition system starts the engine

  19. How a speedometer works

  20. How the liver removes toxins from blood

  21. How the heart pumps blood

  22. How a car differential helps the car turn

  23. How a water faucet controls water flow

  24. How the presidential elections determine the next president

  25. How a quartz watch keeps time

  26. How steam central heating warms large buildings

  27. How a VCR works

  28. How a snare catches small animals

  29. How a car’s gearbox works

  30. How an incinerator works

  31. How a cylinder lock opens with a key

  32. How a television creates pictures

  33. How a helicopter flies

  34. How a ball-point pen writes

  35. How a radio receiver works

  36. How an electric motor changes electricity into movement

  37. How a telephone transmits sound through wires

  38. How piano keys make sounds

  39. How a fireplace works

  40. How a spray-bottle sprays liquids

  41. How a solid-fuel rocket produces thrust

  42. How a manual clutch works

  43. How the aqualung (Scuba-gear) regulates air-pressure

  44. How an Ethernet network allows computers to share files

  45. How a computer mouse controls the pointer on a computer screen

  46. How a transistor works

  47. How a scanner captures images

  48. How the brain coordinates behavior

Further Reading