I recently grew frustrated by some of ChatGPT’s shallow responses, and my frustration helped me to clarify something about what real thinking looks like for humans, and how little real thinking I actually do.
When I was in high school, I had to write a lot of essays and take a lot of tests about various topics I thought were boring and stupid. No matter; I was a clever kid and had lots of data on what essays should look like, and lots of feedback from my teachers to train my responses to any given essay question. So what would happen is that I’d be given a question - “What are the core themes of King Lear?” say - and I would be able to predict the answer that the teacher wanted and output an A+ essay. The whole time, nothing that I would call real thinking was happening. I was never having an original thought, never taking any attempt to understand the subject matter in an honest way, never interrogating what I really thought about something.
Other examples I notice in my own life:
Someone asks a question about a topic (let’s say climate tipping points), and since I recently skim-read a single article about climate tipping points, I confidently state the answer. The whole time, I know deep in my gut that I don’t really know whether my supplied answer is true (or the deeper reason why it’s true).
I take a maths test and am able to apply a sequence of steps to solve an equation, but have a niggling feeling that I’m just barely clinging to the cliff-face of Correctness. All it would take is for the question to be framed in a slightly different way and I would not be able to solve it because I don’t understand the underlying principle that makes them the same sort of question.
I internalise some tweet’s opinion on a topic as my own, without actually interrogating whether I truly believe this.
I am asked by a professor to write about axiology, and I easily write up a smart-sounding and intellectually defensible paper which argues a position that I’m not sure is right.
Some common elements are:
An anticipation (conscious or unconscious) of what belief/opinion/answer/set of ideas that an audience (either real or imagined) would approve of. A grabbing for the convenient answer
A lack of ‘deep’ understanding of underlying principles/reasons/context
A failure to approach the problem with honest humility and firsthand deliberation: what do I really think? What do I really know? If this is true, why is it true?
A resultant feeling in your body that something is hollow/brittle/shaky/inauthentic
When you combine these things you get a failure we can call Parroting. We parrot for all sorts of different reasons: social approval, grades, self-image maintenance, convenience, ideological defensiveness, fear of failure/embarrassment, and even as honest mental shorthand. The problem is that parroting works a great deal of the time, and so we learn to rely on it to get by. This makes failures of parroting much worse. Imagine applying for a job where you write a really slick cover letter and talk up how well you understand the nature of the role. Then imagine you get to the interview and the interviewer is a manager who is a truly clear-eyed and rigorous thinker. What happens? By asking sharp, precise questions aimed at uncovering your true understanding, they can very quickly see through your parroting . All of a sudden, you are forced to confront the fact that you really don’t know what you’re talking about. It’s all superficial.
I hypothesise that this is a partial cause of imposter syndrome: if you never allow yourself to cultivate deep understanding or original thought, then you will be plagued by the feeling that you are a fraud. Conventional solutions to imposter syndrome sometimes say “fake it till you make it” or “you just need to realise that you really are good enough”. But this misses precisely the sense in which you are behaving as an imposter. Only when you confront this habit can you uncover the true competence that you have, the thing that lies underneath all the bullshit.
Notice also what this implies for systems of evaluation. When hiring for a job, or grading students, or performing any of these kinds of tests of merit, there is a constant arms race between the evaluators trying to slice through parroting attempts and the parroters doing everything they can (potentially unconsciously) to avoid having to do any real comprehension. This seems related to Goodhart’s Law to me in ways I haven’t fully thought through. All evaluations are imperfect and end up revealing Skill-At-This-Evaluation more than The-Skill-We-Were-Trying-To-Evaluate. But we can get closer to the latter. Notice for example that a face-to-face interview conducted by a subject expert is much more likely to reveal parroting than most other methods.
But there’s something I’m more interested in than catching parroters in the act, and it’s this: how do we stop parroting?
The first step is Noticing. Saying to ourselves: yes, I have spent my life engaged in such layers of complex parroting that it’s hard to extricate myself. That I have bullshitted to teachers, to professors, to people in my social network. All the while, one is simultaneously bullshitting oneself. But you can catch yourself, mid-parrot. It is possible. There is a feeling there, as you think a cached thought or say the ‘right line’ or take an intellectual shortcut to an answer. Maybe it’s a shameful feeling that you’re just saying something to sound smart. Maybe it’s a discomfort that you haven’t really articulated something, or that something you’ve written is crying out its own incompleteness. Often for me, there’s a kind of ‘coming apart’ feeling in my abdomen as though I am literally compromising my bodily integrity by not being authentic or thoughtful. This is a bodily thing, something you can catch yourself in the middle of doing and stop. The hardest thing – something I still have not been able to do much – is to stop in the middle of parroting and say “sorry, I don’t really know if what I’m saying is true,” ... “sorry, I don’t think I actually believe that. Let me think about this.”
So you’ve noticed your parroting and flagged it, either to yourself or (if you’re brave) to others. You’ve paused to collect yourself. What does a good next step look like?
Maybe it’s something like asking yourself where that thought came from. Were you copying something you heard? Saying something that seemed high status? Writing an answer that felt like the one they were looking for? Repeating an old opinion you have even though you can’t remember where it came from? And it’s worth saying: is this a reasonable thing to claim?
A few things emerge here. The first is that you might notice an impatience or defensiveness in yourself. A kind of yes, I know what I’m talking about or something. Sometimes you might find that actually, you are confident in what you’re saying, but you just needed to check in with yourself about it. But more often I tend to notice that yeah, I’m just trying to give an easy answer, take an intellectual and social shortcut. That’s fine sometimes maybe. But do I want to be doing that? Hacking my way through truth and falsehoods with a kind of indifference to their shape?
Another useful thing is clarifying what you’re uncertain about. Is it that you believe the claim but aren’t sure how it plugs into the rest of your knowledge network? Is it that you are being overconfident? Is it that you aren’t making a claim at all but are instead using words to sound clever or cheer on ‘your team’?
And then, finally, comes the hard part. Zooming out, and seeing the cliff you stand on, empty air beneath you where you were so confidently striding a moment earlier, you have to actually Ask the Question and Really See the terrain.
This skill we could call Seeing has been talked about a lot, by cleverer people than me. It’s related to rationality and epistemic humility and thoughtfulness and doing honest research. But it’s also a kind of way of relating to the world. An opening, a curiosity. It feels like relaxing a clenched fist. You don’t have to pretend anymore, you don’t need to be clever or have the answer. You can be honest. You can ask what is true. And you can see what there is to see.
I’ll close with a few resources that I think are either talking about this exact thing (generally with more comprehensiveness and oomph than I’ve managed here) or doing something usefully adjacent.
Julia Galef’s excellent book The Scout Mindset is very much about Not Parroting.
Eliezer Yudkowsky’s Sequences on LessWrong contain a lot of this sort of thing. Here he shares a great quote from Robert Pirsig’s Zen & the Art of Motorcycle Maintenance about Original Seeing (emphasis mine).
“He'd been having trouble with students who had nothing to say. At first he thought it was laziness but later it became apparent that it wasn't. They just couldn't think of anything to say. One of them, a girl with strong-lensed glasses, wanted to write a five-hundred word essay about the United States. He was used to the sinking feeling that comes from statements like this, and suggested without disparagement that she narrow it down to just Bozeman. When the paper came due she didn't have it and was quite upset. She had tried and tried but she just couldn't think of anything to say. He had already discussed her with her previous instructors and they'd confirmed his impressions of her. She was very serious, disciplined and hardworking, but extremely dull. Not a spark of creativity in her anywhere. Her eyes, behind the thick-lensed glasses, were the eyes of a drudge. She wasn't bluffing him, she really couldn't think of anything to say, and was upset by her inability to do as she was told. It just stumped him. Now he couldn't think of anything to say. A silence occurred, and then a peculiar answer: "Narrow it down to the main street of Bozeman." It was a stroke of insight. She nodded dutifully and went out. But just before her next class she came back in real distress, tears this time, distress that had obviously been there for a long time. She still couldn't think of anything to say, and couldn't understand why, if she couldn't think of anything about all of Bozeman, she should be able to think of something about just one street.
He was furious. "You're not looking!" he said. A memory came back of his own dismissal from the University for having too much to say. For every fact there is an infinity of hypotheses. The more you look the more you see. She really wasn't looking and yet somehow didn't understand this. He told her angrily, "Narrow it down to the front of one building on the main street of Bozeman. The Opera House. Start with the upper left-hand brick."
Her eyes, behind the thick-lensed glasses, opened wide. She came in the next class with a puzzled look and handed him a five- thousand-word essay on the front of the Opera House on the main street of Bozeman, Montana. "I sat in the hamburger stand across the street," she said, "and started writing about the first brick, and the second brick, and then by the third brick it all started to come and I couldn't stop. They thought I was crazy, and they kept kidding me, but here it all is. I don't understand it."
Neither did he, but on long walks through the streets of town he thought about it and concluded she was evidently stopped with the same kind of blockage that had paralyzed him on his first day of teaching. She was blocked because she was trying to repeat, in her writing, things she had already heard, just as on the first day he had tried to repeat things he had already decided to say. She couldn't think of anything to write about Bozeman because she couldn't recall anything she had heard worth repeating. She was strangely unaware that she could look and see freshly for herself, as she wrote, without primary regard for what had been said before. The narrowing down to one brick destroyed the blockage because it was so obvious she had to do some original and direct seeing.”
This blog post by Nabeel Qureshi about understanding, which is the antithesis to parroting. Interestingly he uses the same quote from Pirsig to talk about this.
Joe Carlsmith has several blog posts related to these ideas which I cannot recommend highly enough. I really admire his style of puzzling things through in the open. Most importantly his discussions of sincerity and attunement both feel deeply entwined with Not Parroting in some ways, especially with regards to noticing the felt-sense that something is being dulled or closed off to the world. His continual return to the honest signals of feeling really cut through lot of bullshit for me.
In the interest of practising what I preach, I don’t have a solid understanding of this. That’s not me signalling humility - it’s me not having thought enough or read enough. All I’m doing here is blurting out the bad word, pointing at something that - more and more - I notice in myself. By writing about this I’m trying to unwrap this wound and let in some fresh air and sunlight. I’m trying to be actually honest and thoughtful and uncertain and not just cleverly play the part of a person that is all those things. It’s difficult. But I suppose good things are not always easy.