Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Summarization Without Lobotomization

Here is what happens when you ask an LLM to “summarize this article.” You get back a paragraph that captures the main point, strips away the nuance, removes all uncertainty, and presents a clean narrative where the original had rough edges. The summary is shorter than the source material — congratulations, that is what summarization means. But it is also dumber than the source material, in specific and predictable ways that you should understand before you build your workflow around it.

Default summarization lobotomizes content. It performs a cognitive lobotomy — removing the parts of an argument that are difficult, ambiguous, or in tension with each other, and leaving you with a smooth, confident version of something that the original author was probably not smooth or confident about. If you read only the summary, you will be confidently wrong about the state of knowledge on the topic. If you read the original after the summary, you will be surprised by how much complexity the summary hid.

This chapter is about how to summarize without doing that. How to compress information while preserving the things that matter — the disagreements, the uncertainties, the places where the evidence is thin, the author’s own caveats. This is harder than default summarization, but it is the difference between summaries that make you informed and summaries that make you misinformed.

Why Default Summarization Fails

Default summarization fails for a specific reason that is worth understanding: LLMs are trained to produce fluent, coherent text, and uncertainty is the enemy of fluency. When an original text says “The evidence tentatively suggests X, though two major studies found conflicting results and the mechanism remains unclear,” a default summary will often collapse this into “Research shows X.” The summary is not wrong, exactly — X is what the evidence tentatively suggests. But it is misleading, because it stripped the qualifier “tentatively,” dropped the conflicting studies, and ignored the mechanistic uncertainty. The hedging is gone. The complexity is gone. What remains is a fact-shaped object that looks much more solid than the actual state of knowledge.

This happens because of several compounding tendencies:

Coherence pressure. LLMs produce text that flows well. Hedges, caveats, and contradictions interrupt flow. Given the choice between a clean statement and a qualified one, the model gravitates toward cleanliness.

Central tendency. Summaries trend toward the main point, which is usually the claim the author is making. The supporting evidence, alternative interpretations, and methodological limitations are treated as subordinate details that can be trimmed. But these “details” are often where the real information lives — they tell you how much to trust the main point.

Loss of voice. Every author has a perspective, a level of confidence, and a way of signaling uncertainty. When you summarize their text, these signals are replaced by the model’s default register, which is calm, confident, and authoritative. An author who was clearly uncertain about their conclusions ends up sounding sure of them in the summary.

Missing metadata. Default summaries do not include information about the source: who wrote it, what their credentials are, when it was published, what publication it appeared in, or what their potential biases might be. These contextual details are critical for evaluating information quality and are routinely discarded.

Understanding these failure modes is not a reason to avoid AI summarization. It is a reason to get better at prompting for summaries that do not exhibit them.

Prompting for Better Summaries

The solution is not to ask for summaries. It is to ask for specific kinds of summaries, with explicit instructions about what to preserve.

Technique 1: Preserve the Disagreements

Prompt:

Summarize this text in 300-400 words. Specifically:

- State the author's main claim or argument
- Identify any points where the author acknowledges disagreement or
  alternative views, and include these in the summary
- Note any evidence the author presents that could support a different
  conclusion than the one they reach
- Preserve the author's level of confidence — if they are uncertain,
  the summary should convey that uncertainty

This single prompt dramatically improves summary quality because it explicitly tells the model that the things it would normally strip out — disagreement, uncertainty, alternative interpretations — are the things you want preserved.

Technique 2: Flag the Weak Points

Prompt:

Summarize this text, and in a separate section at the end, note:

1. Where the author's evidence is weakest or most indirect
2. What the author explicitly leaves uncertain or unresolved
3. Any claims that are stated confidently but not well-supported
   within the text
4. What a knowledgeable critic would challenge first

This produces a two-part output: a summary and a credibility assessment. The summary tells you what the text says; the assessment tells you how much to trust it. Together, they give you more useful information than either would alone.

Technique 3: Multi-Perspective Summary

Prompt:

Summarize this text three times:

1. First, summarize it as the author would want it summarized — their
   intended takeaway
2. Second, summarize it as a skeptical peer reviewer would — what are
   the limitations and open questions?
3. Third, summarize it as someone from [a different field or
   perspective] would — what looks different from the outside?

Keep each version to 150-200 words.

This is more expensive in terms of output length, but it gives you a triangulated view that is much closer to what you would get from actually reading the full text. The three perspectives illuminate different aspects of the content and help you avoid the tunnel vision that comes from a single summary.

Technique 4: Structured Metadata Summary

Prompt:

Provide a structured summary of this text with the following fields:

Source: [publication name and date]
Author: [name and relevant credentials/affiliation]
Type: [research paper / opinion / reporting / analysis / review]
Main Claim: [one sentence]
Key Evidence: [2-3 most important pieces of supporting evidence]
Counterevidence or Limitations: [anything that qualifies the main claim]
Author's Confidence Level: [high / moderate / hedged / speculative]
Potential Bias: [any obvious perspective or interest that might affect
the analysis]
Reading Recommendation: [skim / read in full / deep read with notes /
skip]
One-Paragraph Summary: [the actual summary, 150-200 words]

This template produces summaries that are immediately actionable. You can scan the metadata fields in 10 seconds and decide whether the full summary is worth reading. The structured format also makes it easy to compare multiple summaries side by side.

Technique 5: Layered Summarization

Sometimes you need different levels of detail for different purposes. A headline for scanning, an executive summary for quick understanding, and a detailed brief for reference.

Prompt:

Provide a three-level summary of this text:

Level 1 — Headline (max 15 words): A single sentence capturing the
core finding or argument.

Level 2 — Executive Summary (50-75 words): The main claim, the key
evidence, and the primary limitation or caveat. Written for someone
who needs to decide whether to read further.

Level 3 — Detailed Brief (250-400 words): A full summary preserving
nuance, disagreements, methodology notes, and the author's own
expressed uncertainty. Written for someone who needs to understand
the content well enough to discuss it intelligently without having
read the original.

Layered summaries are particularly useful for building a personal knowledge base. You can store Level 1 as a searchable index, Level 2 for quick reference, and Level 3 for when you need to recall the details.

Summarizing Different Types of Content

Not all content should be summarized the same way. A research paper, a news article, and a long-form opinion piece have different structures, different relationships to truth, and different failure modes when summarized carelessly.

Research Papers

Research papers have a built-in structure: abstract, introduction, methods, results, discussion, conclusion. Default summarization often just paraphrases the abstract, which misses the methodological details that determine whether the results are trustworthy.

Prompt for research papers:

Summarize this research paper with attention to:

1. Research question: What exactly were they testing?
2. Methodology: How did they test it? What was the sample size?
   What were the key methodological choices?
3. Results: What did they find? Include effect sizes and confidence
   intervals if available.
4. Limitations: What limitations do the authors acknowledge?
   What additional limitations are apparent?
5. Context: How does this fit into the broader research landscape?
   Does it confirm or challenge existing findings?
6. Practical implications: What would change if we took these
   findings seriously?

Do not inflate the certainty of the findings. If the results are
preliminary, say so. If the sample is small, say so. If the effect
size is modest, say so.

The last instruction is critical. LLMs have a strong tendency to make research findings sound more definitive than they are. “The study found that X increases Y” is much more confident than what most papers actually demonstrate, which is usually something like “In this sample, under these conditions, we observed a statistically significant but moderate association between X and Y.”

News Articles

News articles are reports about events, and their value depends heavily on the quality of the reporting: who the sources are, whether claims are independently verified, and what context is provided.

Prompt for news articles:

Summarize this news article with attention to:

1. What happened: The core facts being reported
2. Sources: Who is the information coming from? Named or anonymous?
   How many independent sources?
3. What is claimed vs. what is verified: Distinguish between
   confirmed facts and allegations/claims
4. Missing context: What background information would help a reader
   evaluate this story?
5. What we don't know: What questions does this article leave
   unanswered?

Keep opinions attributed to their sources rather than presenting them
as facts.

Long-Form Arguments

Opinion pieces, essays, and analysis pieces are making arguments rather than reporting facts. Summarizing them requires capturing both the argument and its persuasive strategy.

Prompt for argumentative pieces:

Summarize this argument with attention to:

1. Thesis: What is the author arguing?
2. Key premises: What are the 2-3 most important claims the argument
   depends on?
3. Evidence: What evidence does the author provide for each premise?
4. Logical structure: How does the argument hold together? Are there
   any gaps or leaps?
5. What the author is responding to: What opposing view or conventional
   wisdom is this argument pushing against?
6. Strongest point: Where is the argument most convincing?
7. Weakest point: Where is it most vulnerable to counterargument?

This summary tells you not just what someone thinks, but why they think it and how well-supported their thinking is. That is the difference between a summary that informs and one that merely abbreviates.

Building a Summarization Pipeline

If you process a significant volume of information regularly, it is worth building a consistent pipeline rather than crafting individual prompts each time.

Step 1: Define Your Summary Template

Based on the techniques above, create 2-3 summary templates that cover the types of content you most frequently process. For example:

  • Quick Assessment Template: Metadata + Level 1-2 summary. Use for triage — deciding whether something is worth more time.
  • Full Summary Template: Structured metadata + Level 3 summary + weak points section. Use for content you need to understand but will not read in full.
  • Research Paper Template: The specialized research paper prompt above. Use for academic papers.

Save these templates somewhere accessible. A text file, a note in your note-taking app, a snippet in your text expander. The point is that you should not be re-inventing your summary prompt every time.

Step 2: Create a Metadata Standard

Every summary you produce should include a minimum set of metadata:

  • Source: Publication name and URL
  • Author: Name and, if easily available, affiliation
  • Date: Publication date
  • Date summarized: When you created the summary
  • Content type: Research / news / opinion / analysis / review / tutorial
  • Confidence assessment: How reliable does this source appear to be?
  • Relevance to: Which of your active priorities does this relate to?

This metadata turns your summaries into a searchable, sortable knowledge base. Without it, they are just a pile of disconnected paragraphs.

Step 3: Establish a Verification Checkpoint

Not every summary needs full verification. But you should have a clear rule about when verification is required:

Always verify when:

  • The summary will inform a decision
  • You will share the summary with others
  • The content makes claims that seem surprising or counterintuitive
  • The source is unfamiliar to you
  • Specific statistics or data points are central to the summary

Verification can wait when:

  • The summary is for your personal reference only
  • The content is from a highly trusted source you have verified before
  • The claims are consistent with your existing knowledge
  • You are summarizing for orientation, not for action

Step 4: The “Summarize Then Verify” Workflow

Here is the complete workflow for processing a piece of content through your summarization pipeline:

1. Identify content type (research, news, opinion, etc.)
2. Select appropriate summary template
3. Generate summary using LLM
4. Scan the summary — does it pass the smell test?
   - Are claims plausible?
   - Does the confidence level seem calibrated?
   - Are there any surprising claims that need checking?
5. If verification needed:
   a. Identify 2-3 key claims to check
   b. Cross-reference with independent sources
   c. Note any discrepancies in the summary
   d. Add a verification note to the summary
6. File the summary with metadata
7. Tag with relevant priority areas

This workflow takes 5-10 minutes for a standard article with no verification needed, and 15-20 minutes when verification is required. Compare that to the 20-40 minutes it would take to read the full article and create manual notes. The time savings compound significantly when you are processing multiple items per day.

When Summarization Is Not Enough

There are times when summarization, no matter how well-executed, is not appropriate. You need to read the whole thing. Recognizing these situations is as important as having good summarization skills.

Read the full text when:

The source is a primary document relevant to a decision. If you are evaluating a contract, a policy, a technical specification, or a regulatory document, do not rely on a summary. The details — the specific wording, the exceptions, the fine print — are the entire point. A summary of a contract is not a substitute for reading the contract.

The argument’s value is in its reasoning, not its conclusions. Some texts are worth reading not for what they conclude but for how they get there. A well-reasoned analysis teaches you something about how to think, not just what to think. Summarization captures the what; it cannot capture the how.

You are going to be accountable for the content. If you will be presenting this information, answering questions about it, or making decisions that others will scrutinize, read the original. “My LLM summary said…” is not a defensible basis for important decisions.

The content is in your core area of expertise. In your primary field, you should be reading the actual work, not summaries. You have the background to extract nuance that no summarization prompt can capture, and staying close to the primary literature is how you maintain and develop your expertise.

The content is genuinely enjoyable. If you are reading for pleasure, learning, or intellectual stimulation, summarizing defeats the purpose. Some things are worth reading slowly, not because you have to, but because the experience of reading them is valuable.

You have a nagging feeling the summary is missing something. Trust this instinct. If a summary feels too clean, too simple, or too conveniently aligned with what you expected, read the original. Your subconscious pattern-recognition is picking up on something, even if you cannot articulate what.

Common Summarization Anti-Patterns

These are the mistakes I see most frequently, including in my own practice:

The Telephone Game

Summarizing a summary. You read someone’s thread summarizing an article, then ask the LLM to summarize the thread. You are now two levels of compression away from the source, and each level of compression strips more nuance. If something is important enough to summarize, go back to the original source.

The Headline Trap

Treating Level 1 summaries (headlines) as if they were Level 3 summaries (detailed briefs). A headline tells you the topic and the main claim. It tells you nothing about the evidence, the caveats, or the context. If you are making decisions based on headline-level understanding, you are not actually informed — you just have the vibe.

The Confirmation Summary

Unconsciously adjusting your summary prompts to produce results that confirm what you already believe. “Summarize this article, focusing on the evidence that supports [your existing view]” is technically a valid prompt, but it produces a biased summary. If you notice yourself consistently prompting in ways that filter out unwelcome information, you have a problem that is bigger than summarization technique.

The Archive Graveyard

Generating beautiful, well-structured summaries and then never looking at them again. The summary pipeline is only useful if the summaries are retrievable and actually retrieved. If your summary archive has grown to 500 items and you have never gone back to reference one, you are doing elaborate busywork that feels productive. Either build a system for actually using your summaries (tags, search, periodic review) or accept that some things do not need to be summarized at all.

The Nuance Restoration Fantasy

Believing that a good enough prompt will make a summary as nuanced as the original. It will not. Summarization is by definition a lossy compression. The best summary preserves the most important nuance, but it always loses something. Do not let the quality of your summarization pipeline trick you into thinking you understand a topic deeply when you have only read summaries.

Practical Templates You Can Use Today

Here are four copy-paste-ready templates. Modify them to fit your needs, but they work well as starting points.

Template A: Quick Triage Summary

Provide a triage summary of this text:

FORMAT:
- Type: [research/news/opinion/analysis]
- Relevance: [who would find this most useful and why]
- Headline: [max 15 words]
- Key claim: [one sentence]
- Confidence: [well-supported / partially-supported / speculative]
- Time investment: [skim (2 min) / read (10 min) / deep read (30+ min)]
- One-paragraph summary: [100-150 words, preserving key caveats]

Template B: Decision-Support Summary

Summarize this text as if I need to make a decision based on it.

INCLUDE:
1. The core finding or recommendation (2-3 sentences)
2. The strongest evidence supporting it
3. The most significant evidence or argument against it
4. What the text explicitly does NOT address that might affect
   the decision
5. The author's potential biases or conflicts of interest, if apparent
6. Your assessment: if I had to act on this alone, what would I be
   risking?

Be direct. Flag anything I should verify before relying on this.

Template C: Literature Review Summary

Summarize this academic paper for inclusion in a literature review:

1. Full citation: [format as APA]
2. Research question
3. Methodology (including sample characteristics and key design choices)
4. Key findings (with effect sizes where applicable)
5. Authors' stated limitations
6. Additional methodological concerns
7. How this relates to [your specific research question]
8. Key quotes worth preserving (with page numbers if available)
9. Studies cited by this paper that I should also read

Maintain the authors' level of certainty — do not overstate findings.

Template D: Comparative Summary

I'm going to give you [N] texts on the same topic. Summarize them
comparatively:

1. Where do they agree? List the points of consensus.
2. Where do they disagree? For each disagreement, state each text's
   position and the evidence each cites.
3. What does each text include that the others do not?
4. Which text provides the strongest evidence for its claims? Why?
5. What is the overall state of knowledge based on these texts
   together? Where is it solid and where is it uncertain?

This last template is especially powerful for quickly synthesizing multiple sources on the same topic. Instead of reading all five articles about a topic and mentally tracking where they agree and differ, you process them through this template and get a structured comparison. It is not a substitute for reading all of them — but it tells you which ones are most worth reading in full.

Building the Habit

Summarization technique only matters if you actually use it consistently. Here are some practical suggestions for making it habitual:

Start with one template. Do not try to implement the full pipeline immediately. Pick the template that matches your most common use case and use it for a week. Add complexity only after the basic practice is established.

Keep your templates accessible. If you have to reconstruct a prompt from memory each time, you will gradually drift back to “just summarize this.” Store your templates in a text expander, a pinned note, or wherever you store things you need to access quickly and frequently.

Review your summaries weekly. During your weekly triage review (Chapter 9), spend five minutes looking at the summaries you produced that week. Were they useful? Did you reference any of them? Did you find that any were misleading when you later read the original? Adjust your templates based on what you learn.

Pair summarization with verification. Make it a habit to verify at least one claim per summary. This does not add much time, and it trains your instinct for when a summary is reliable and when it needs scrutiny.

The goal is not to summarize everything. The goal is to make your summarization practice good enough that, when you do summarize, the result is actually useful — informing your understanding rather than merely abbreviating text while quietly discarding everything that made it worth reading in the first place.