📚AI Companies Secretly Buying and Destroying Millions of Books
Physical books vanish as AI hoards data
TL;DR
AI firms are secretly buying millions of physical books to train their models, then destroying them. This ensures they have unique training data but raises concerns about knowledge monopolization.
AI companies are quietly purchasing and scanning millions of physical books before destroying them all. Why? To get access to untouched pre-2022 data for model training. But this also means the public loses out on these irreplaceable resources. For instance, Anthropic's Project Panama spent tens of millions buying and destroying books since early 2024. The destruction is cheaper than lossless scanning but ensures AI firms have exclusive digital copies. This monopolizes knowledge, leaving everyone else behind. So what can be done? Shadow libraries are stepping up to digitize as much content as possible.
Key Points
Anthropic's Project Panama spent $150 million since early 2024 to buy and destroy books for Claude LLM training.
AI firms are scanning physical books before destroying them, ensuring they have unique pre-2022 data for model training.
Shadow libraries aim to digitize 10 million pieces of invaluable content if every person scans a book as a volunteer.
Recognition and lifetime membership can be awarded by Anna's Archive for small uploads and large-scale scanning efforts.
Since early 2025, AI-generated content now accounts for over half of newly published internet material.
Why It Matters
If you're working on an AI project that relies on historical data, the unique training sets AI companies are hoarding could make your work obsolete. For example, if Anthropic has exclusive access to pre-2022 books for Claude LLM, other projects might struggle to compete without similar resources.
Frequently Asked Questions
Why does this matter?
If you're working on an AI project that relies on historical data, the unique training sets AI companies are hoarding could make your work obsolete. For example, if Anthropic has exclusive access to pre-2022 books for Claude LLM, other projects might struggle to compete without similar resources.
What happened?
AI firms are secretly buying millions of physical books to train their models, then destroying them. This ensures they have unique training data but raises concerns about knowledge monopolization.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,293 builders reading daily.