If approved, the settlement would be the largest in the history of American copyright cases, according to a lawyer for the authors behind the lawsuit.
Anthropic, a major artificial intelligence company, has agreed to pay at least $1.5 billion to settle a copyright infringement lawsuit filed by a group of authors who alleged the platform had illegally used pirated copies of their books to train large-language models, according to court documents.
“If approved, this landmark settlement will be the largest publicly reported copyright recovery in history, larger than any other copyright class action settlement or any individual copyright case litigated to final judgment,” said Justin Nelson, a lawyer for the authors.
The lawsuit, filed in federal court in California last year, centered on roughly 500,000 published works. The proposed settlement amounts to a gross recovery of $3,000 per work, Nelson said in a memorandum to the judge in the case.
It’s true that a new model can be initialized from an older one, but it will never outperform the older one unless it is given actual training data (not necessarily the same training data used previously).
Kind of like how you can learn ancient history from your grandmother, but you will never know more ancient history than your grandmother unless you do some independent reading.
I think we’re in agreement with each other? The old model has the old training data, and then you train a new one on that model with new training data, right?
No, the old model does not have the training data. It only has “model weights”. You can conceptualize those as the abstract rules that the old model learned when it read the training data. By design, they are not supposed to memorize their training data.
To outperform the old model, the new model needs more than what the old model learned. It needs primary sources, ie the training data itself. Which is going to be deleted.
I expressed myself poorly, this is what I meant - it has the “essence” of the training data, but of course not the verbatim training data.
I wonder how valuable in relative terms the old training data is to the process, compared to just the new training data. I can’t answer it, but it would be interesting to know.
A new model needs training data, it doesn’t matter if the data is new or old. But generally, a more advanced model needs more training data, so AI devs generally need at least some new training data.