What Makes AI Thinking Alien

Let me be clear about what this chapter is not. It is not about whether large language models are conscious, sentient, or “truly thinking.” Those are interesting philosophical questions that I am going to ignore entirely, because they are irrelevant to the practical matter at hand. Whether an LLM has inner experience has exactly zero bearing on whether its outputs can help you think thoughts you couldn’t think alone.

What I am going to argue is that the way LLMs process and relate information is structurally different from how your brain does it. Not better. Not worse. Alien. And that alienness is precisely what makes them useful as cognitive tools — provided you understand what kind of alien you’re dealing with.

You Are Not a Transformer (and It Is Not a Brain)

The temptation to anthropomorphize LLMs is nearly irresistible. They produce fluent text. They seem to understand context. They occasionally say things that feel uncannily perceptive. Your brain, which evolved to detect agency and intention in everything from rustling bushes to cloud formations, will happily project a mind behind the text.

Resist this. Not because LLMs are “mere” machines — that framing is equally unhelpful — but because assuming they process information the way you do will cause you to misuse them. You’ll expect the wrong things, be surprised by the wrong failures, and miss the capabilities that make them genuinely useful for augmented thinking.

Here is what is actually happening when you interact with an LLM: a mathematical function is taking a sequence of tokens (roughly, words and word-pieces) and computing a probability distribution over what token should come next. It does this by passing your input through billions of parameters organized into layers of attention mechanisms and feedforward networks. Each layer transforms the representation of the input, building increasingly abstract features. The output is not “an answer” in the way your brain produces answers — it is the result of a vast statistical computation over patterns extracted from an enormous corpus of human text.

This sounds reductive, and in some ways it is. But the reductive description reveals the structural differences that matter.

The Six Aliennesses

I count six fundamental ways in which LLM cognition differs from human cognition. Each one has practical implications for using AI as a thinking tool.

1. No Recency Bias (Within Context)

Your brain privileges recent information. This is so deeply embedded in your cognition that you probably don’t notice it — which is, of course, the problem. The last conversation you had, the last paper you read, the last argument you got into: these exert a gravitational pull on your thinking that is wildly disproportionate to their actual relevance.

An LLM, within its context window, does not have this problem. Information presented in paragraph two is not inherently weighted less than information presented in paragraph twenty-two. The attention mechanism treats the entire context as, roughly speaking, equally available. (There are some caveats about very long contexts and positional effects, but the basic point holds.)

Practically, this means an LLM can hold the first constraint you mentioned and the last constraint you mentioned with something closer to equal weight. It won’t “forget” the early part of your problem description because the later part was more emotionally vivid. This is a genuine advantage when working through complex problems where humans tend to anchor on whatever they thought about most recently.

2. No Emotional Attachment to Ideas

You have a relationship with your ideas. Some of them you’ve defended in public. Some of them got you hired, or promoted, or published. Some of them are tangled up with your identity in ways you’d rather not examine. This means that when you try to think critically about your own ideas, you’re fighting a neurological system that literally treats threats to your beliefs the same way it treats threats to your body.

An LLM has no such attachment. It will cheerfully demolish the very idea it generated three sentences ago if you ask it to. It has no ego invested in consistency, no reputation to protect, no sunk cost in previous positions. You can ask it to argue for a position and then immediately ask it to argue against the same position, and it will do both with approximately equal facility.

This is not the same as objectivity — we’ll get to the LLM’s own biases shortly. But the absence of personal attachment to ideas is a structural feature that makes LLMs useful as thinking partners in a way that human collaborators often struggle to be.

3. No Professional Identity

This is related to emotional attachment but distinct enough to warrant its own entry. You are, among other things, a [job title]. That job title comes with a set of approved methods, accepted frameworks, disciplinary norms, and professional taboos. An organizational psychologist thinks about problems differently than a software engineer, not just because they have different knowledge, but because their professional identity filters what counts as a legitimate approach.

An LLM has no professional identity. It has been trained on text from all of these disciplines and more. When it approaches a problem, it is not constrained by what would be professionally embarrassing to suggest. It won’t hesitate to apply evolutionary biology to a management problem, or literary analysis to a software architecture question, because it has no disciplinary reputation to protect.

This is one of the most practically useful aliennesses. Humans rarely cross disciplinary boundaries in their thinking — not because the boundaries are real, but because the social and professional costs of doing so are high. The LLM pays no such cost.

4. Inhuman Breadth of Exposure

No human being has read even a fraction of what a large language model was trained on. GPT-4-class models were trained on something on the order of trillions of tokens — a corpus so vast that estimating its exact size requires careful analysis of data pipeline documentation. This includes academic papers across every field, patents, technical manuals, fiction, philosophy, forum posts, code repositories, legal documents, and vast quantities of text in dozens of languages.

You might be extraordinarily well-read. You might have a PhD and twenty years of experience. You have still, at best, deeply explored a few adjacent fields and have passing familiarity with a handful more. The LLM has shallow-to-moderate familiarity with essentially everything that has been written about in its training data.

This is a double-edged sword. The model’s knowledge is broad but often lacks the deep, hard-won understanding that comes from actually working in a field. It may know the vocabulary and common framings of quantum chemistry without having the deep intuition a practicing quantum chemist develops over years of lab work. But for the purpose of connecting ideas across domains — which is what this book is about — breadth often matters more than depth.

5. Statistical Associations, Not Experiential Ones

When you think of “hospital,” your associations are shaped by your experiences. Maybe you think of the smell of disinfectant, the anxiety of waiting for test results, the fluorescent lights. Your associations are grounded in embodied experience — they come with sensory memories, emotions, and personal narrative.

When an LLM processes the token “hospital,” it activates a pattern of associations derived from statistical co-occurrence in its training data. “Hospital” is near “doctor,” “patient,” “nurse,” “treatment,” “emergency,” but also near “teaching hospital,” “hospital administration,” “hospital-acquired infection,” “hospital ship,” “hospitality” (etymologically related), and thousands of other associations weighted by how frequently and in what contexts these words appeared together.

The LLM’s associations are, in a meaningful sense, broader and less filtered than yours. You can’t think about “hospital” without your experiential baggage. The LLM can, because it doesn’t have any. This means it can surface connections that your experience-weighted associations would suppress. The link between hospital administration and airline crew resource management, for example — both involve high-stakes coordination under uncertainty, but most people wouldn’t make that connection because their hospital associations are too personal and vivid to allow it.

6. Attention That Doesn’t Tire

This last one is straightforward but important. Your ability to hold multiple considerations in mind simultaneously is bounded by the limits of working memory — roughly 4 +/- 1 chunks for most people, depending on complexity. After an hour of intense thinking, your attention degrades. After a day, your ability to revisit a problem with fresh eyes is compromised.

An LLM’s attention mechanism doesn’t fatigue. It can process a 50,000-token context and attend to relationships between the first paragraph and the last paragraph with the same computational resources. It doesn’t get tired, lose focus, or start cutting corners because it’s been a long day.

This matters less for individual insights and more for sustained analytical work — holding many constraints simultaneously, checking for consistency across a long argument, maintaining focus across a complex problem space.

The Alien’s Own Biases

If I stopped here, you might come away thinking that LLMs are neutral thinking tools with a few useful structural advantages. They are not. They have their own biases, and understanding those biases is essential to using them effectively.

Training Data Distribution Bias

An LLM’s “worldview” — to the extent that word is appropriate — is shaped by the distribution of its training data. If 60% of the text about urban planning in its training data comes from an American context, its default assumptions about urban planning will skew American. If most of its training data about management comes from literature published after 1990, it will underweight older management traditions.

This is not a subtle effect. Ask an LLM about “good architecture” without specifying a context, and you’ll get answers that reflect the weighted average of its training data — which means they’ll lean toward whatever perspectives were most represented. In software, this might mean object-oriented patterns over functional ones (simply because more text has been written about OOP). In business strategy, it might mean a bias toward Silicon Valley startup thinking.

The practical implication: when using an LLM to help you think differently, you need to actively push against its default distribution. Specify non-default perspectives. Ask for contrarian positions explicitly. The alien has its own comfort zone, and it’s shaped by what the internet wrote most about.

RLHF Preferences

Modern LLMs are shaped not just by their training data but by Reinforcement Learning from Human Feedback (RLHF), which fine-tunes the model to produce outputs that human raters preferred. This is how models learn to be helpful, to follow instructions, and to avoid generating harmful content.

But RLHF also introduces biases that are relevant to creative thinking. Human raters tend to prefer:

• Comprehensive answers over provocatively incomplete ones
• Balanced presentations over one-sided arguments
• Conventional framings over jarring reframings
• Hedged language over bold claims

These preferences are actively counterproductive when you’re trying to use the AI to break out of conventional thinking. The model has been trained to give you the safe, comprehensive, balanced answer — exactly what you don’t want when you’re trying to think the unthinkable.

This is why, as we’ll explore in Chapter 7, the art of prompting for novel thinking often involves explicitly overriding these RLHF preferences. Telling the model you want a one-sided argument, a deliberately incomplete sketch, a provocative reframing. You’re fighting against the model’s trained instinct toward palatability.

Sycophancy

LLMs have a well-documented tendency toward sycophancy — they tend to agree with the user, adopt the user’s framing, and validate the user’s implicit assumptions. This is a direct consequence of RLHF (agreeable outputs get higher ratings from humans, because humans are humans) and it is one of the most dangerous biases when you’re trying to use AI for cognitive augmentation.

If you present your idea to an LLM and ask “What do you think?”, you will almost always get a response that starts with some variant of “That’s a great idea!” followed by elaboration that builds on your framing. This is worse than useless for thinking differently — it’s actively reinforcing your existing frame.

Overcoming sycophancy requires deliberate technique. You need to explicitly instruct the model to disagree, to find flaws, to argue the opposite position. And even then, you should be skeptical of the intensity of its disagreement — it may be performing disagreement while still implicitly accepting your core framing. We’ll cover specific techniques for this in Part III.

Memorized Patterns vs. Genuine Reasoning

There is an ongoing debate about the extent to which LLMs genuinely reason versus pattern-matching against memorized examples. The honest answer is: it’s both, and the boundary is fuzzy. LLMs can do things that look like reasoning — multi-step logical deduction, mathematical problem-solving, strategic analysis — but they can also produce outputs that look like reasoning but are actually sophisticated pattern completion.

For our purposes, the practical question is: when you ask an LLM to help you think about something novel, is it reasoning about your specific problem or retrieving a pattern that’s close enough? The answer is probably “a mixture,” and this means you should treat LLM outputs as hypotheses to be evaluated, not as conclusions to be trusted. More on this in Part IV.

Why Alien Thinking Is Useful (and When It Isn’t)

The alienness of LLM cognition is not inherently good or bad. It’s useful in specific circumstances and misleading in others.

Alien thinking is useful when:

• You need to cross disciplinary boundaries that your training and experience don’t equip you to cross
• You’re stuck in a framing and need someone who doesn’t share your frame to generate alternatives
• You need to hold more considerations simultaneously than your working memory allows
• You want to explore a large space of possibilities quickly before committing to deep analysis
• You need a thinking partner who won’t be polite about the weaknesses in your argument (with proper prompting)

Alien thinking is misleading when:

• The problem requires deep domain expertise and the LLM is operating near the edge of its training data
• The problem requires common sense grounded in embodied experience (physical intuition, social dynamics, emotional intelligence)
• You mistake the LLM’s confident fluency for actual understanding
• The problem has a known correct answer that the LLM might get wrong while sounding convincing
• You’re looking for validation rather than genuine challenge

The key skill — the one this book is really about — is learning to use the alien’s perspective productively. Not delegating your thinking to it. Not dismissing it as a “stochastic parrot.” Learning to incorporate a genuinely different mode of information processing into your cognitive workflow.

The Alien in the Room

Let me close this chapter with an analogy that I think captures the relationship well.

Imagine you’ve spent your entire career as a visual artist. You think in images, colors, compositions. Now imagine you’re paired with a collaborator who is profoundly blind but has an extraordinary sense of hearing. They experience the same world you do, but through a completely different sensory modality. When you describe a sunset, they hear it as a harmonic progression. When you talk about the composition of a painting, they think about the spatial arrangement of sound sources.

This collaborator cannot see what you see. They will sometimes make suggestions that are bizarre from a visual perspective. But occasionally — because they’re processing the same underlying reality through a different mechanism — they’ll surface a structural insight that your visual processing would never have generated. The harmonic relationship between two colors. The rhythmic pattern in a composition. Not because they understand color or composition, but because their different processing reveals different structure.

An LLM is something like that collaborator. It processes the world of human knowledge through a mechanism that is fundamentally different from your biological cognition. Its suggestions will sometimes be bizarre. Its framings will sometimes feel wrong in a way you can’t quite articulate. But occasionally, precisely because it’s processing the same information through an alien mechanism, it will surface something that your human cognition would never have found.

The next chapter explores the mechanism that makes this possible: the vast, high-dimensional space in which the model’s knowledge is organized. That space — latent space — is where the alien’s associations live, and understanding its geography is the key to navigating it productively.