# How to fact-check an AI-drafted book

List every claim a reader could check, then trace each one to a source you have opened yourself. Start with numbers, quotations, names and citations, where invented details do the most harm. If you can’t confirm a claim, cut it or rewrite it as your opinion. An AI can help you find what to check, but its say-so isn’t a source.

Last reviewed 30 October 2026. Web page: https://chapbook.ink/guides/fact-check-an-ai-drafted-book/

## Why AI drafts need this

A language model writes what is likely, which isn’t the same as what is true. It can produce a plausible statistic or a citation to a paper that doesn’t exist, in the same even tone as everything else.

One study measured this directly. Walters and Wilder had ChatGPT write short literature reviews on 42 topics and checked the 636 citations. In their words, “55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated”, and “43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors”. Those were early-2023 models, and newer ones will differ. Take away the shape of the problem instead of the number: a citation can be invented outright, and a real one can still be wrong in its details.

## What to look for, and how to check it

| Claim | What goes wrong | How to check |
|---|---|---|
| **Statistics** | The number is invented, or real but old, or from a different group | Find the original study or dataset. Note its year and who it measured |
| **Quotations** | The words are close but not what was said, or the wrong person said them | Find the exact sentence in the original text |
| **Citations** | The paper doesn’t exist, or the authors, year or title are wrong | Look up the title and open the paper. Read what it actually claims |
| **People and companies** | Wrong job, wrong date, an invented event | Use the person’s or company’s own site, or a reliable news source |
| **Laws, rules and prices** | True once, changed since | Use the official page, and put the date you checked in the book |
| **“Studies show”** | No study behind it | Name the study or cut the phrase |

Examples and anecdotes need the same care. If the draft describes “a client” or “a coach in Denver”, either replace it with something that really happened, or say plainly that the example is made up.

## A method for what you can’t place

Mike Caulfield’s SIFT is built for checking things found online, and it works well here. His four moves are: **Stop**; **Investigate the source**; **Find better coverage**; and **Trace claims, quotes, and media back to the original context**. The last is the one AI drafts need most. A good claim usually has a traceable origin, and if every search leads back to the same unsourced line, treat that as your answer.

## Your name is on it

KDP’s guidelines are direct: “You are responsible for verifying that all AI-generated and/or AI-assisted content adheres to all content guidelines, including by complying with all applicable intellectual property rights.” That sentence is about content rules and rights, but readers apply the same standard to facts. A nonfiction book that gets one checkable thing wrong makes readers doubt the rest of it.

## Where Chapbook fits

Chapbook can narrow the job, not do it. You can add up to 10 sources to a book’s notes, and the drafting step may cite only those by their keys; a citation to anything else is stripped out. Web research overviews read part of a long source and say so, so check the original before relying on a quotation.

The **Polish** read-through then looks over the whole book and flags claims of fact or figures with no supporting source, citations that don’t match the source they name, and contradictions between chapters. It quotes the sentence, and where one small edit would fix it, offers a change for you to accept or reject. It is an AI reading your book, so treat its list as a place to start. It only knows your sources, and it can miss things.

The sample on this page was drafted with Chapbook and left unedited. Its brief asks for a method, so most of what it says is advice to weigh and not fact to check. Try the steps above on it anyway.

## What to do

1. **Mark every checkable claim.** Go through each chapter and highlight numbers, dates, names, quotations, citations, and any sentence that says what research or experts found. Don’t check yet. Just list them.
2. **Check the riskiest first.** Start with anything about health, money or the law, then figures and quotes, then named people and companies. If time runs short, you have covered what could hurt a reader.
3. **Open the original.** Find the study, report or book itself and read the passage. A blog post that repeats the number isn’t the source, and neither is the model that wrote the sentence.
4. **Keep, fix or cut.** If the source agrees, keep the claim and note the link. If it says something else, fix it. If you can’t find it, cut it or rewrite it as your own view.
5. **Have a person read it.** Give the checked draft to someone who knows the subject, and ask what looks wrong. In a regulated field, that reader should be a qualified professional.

## Made with Chapbook: The Pricing Playbook

The Pricing Playbook: How founders set a price they can defend, by Jane Founder. Nonfiction book, 9,862 words, 43 pages. A sample generated from one brief and exported unedited; it is not a customer's book. Read every page: https://chapbook.ink/examples/nonfiction/

The brief:

> A repeatable method for pricing software: customer interviews, a value metric, tiering, and setting numbers you can defend. Promise: A repeatable method for setting a price you can defend to customers, investors and yourself, validated by ten customer interviews in a week. Reader: A founder who priced by guessing or by copying a competitor and now has to defend or change the number. Voice: Direct second person with occasional first person; worked examples with fictional companies; tables where they clarify; no invented statistics or named studies: reason from mechanics and examples.

The outline the planner proposed:

1. Introduction (620 words)
2. Why Guessing Fails (1,014 words)
3. What You Are Actually Selling (1,004 words)
4. Interview Ten Customers (1,088 words)
5. Choose the Value Metric (979 words)
6. Draft Three Tiers (1,076 words)
7. Set the Numbers (1,130 words)
8. Say the Price Out Loud (1,063 words)
9. Revisit Every Quarter (994 words)
10. Conclusion (384 words)
11. Appendix: The Interview Script (475 words)

## Questions

### How often does AI invent citations?

In a 2023 study, 55% of GPT-3.5’s citations and 18% of GPT-4’s were fabricated. Those are early-2023 models and rates will differ today, so check every citation anyway.

### Can I ask the AI whether a fact is true?

You can ask it where to look, but its answer isn’t a source. A model can confirm its own mistake with confidence. Open the original document and read the passage yourself.

### What should I check first in an AI-drafted book?

Claims that could harm a reader, on health, money or the law. After those, statistics, quotations, citations and named people or companies, since these are the details readers can look up.

### Am I responsible for errors in an AI-drafted book?

On KDP, you are responsible for verifying that AI content adheres to its content guidelines, including intellectual property rights, as checked on 29 October 2026. Your readers will hold you to the rest.

### Does Chapbook fact-check my book?

Not by itself. Its Polish read-through flags claims with no supporting source and citations that don’t match. That narrows the list; it doesn’t prove anything true, and the check is still yours.

## Sources

- [Walters and Wilder, “Fabrication and errors in the bibliographic citations generated by ChatGPT”, Scientific Reports, 7 September 2023](https://pmc.ncbi.nlm.nih.gov/articles/PMC10484980/)
- [Mike Caulfield, “SIFT (The Four Moves)”, 19 June 2019](https://hapgood.us/2019/06/19/sift-the-four-moves/)
- [Amazon KDP, “Content Guidelines”](https://kdp.amazon.com/en_US/help/topic/G200672390)