GPT3 API vs. Reality
Natural language book finding on
LibraryThing.
Tim Spalding, LibraryThing
March 28, 2023

My Business: Book Lovers and Libraries
LibraryThing
(Goodreads cloned us)
TinyCat
(Libary catalogs for small
schools, churches, etc.)
Syndetics
Unbound
(Makes “real” library catalogs
better.)

Everyone is talking about AI
• Right?

The “Holy Shit” Moment
• I’ve long been a skeptic.
• Was I right?
• Called an all-company meeting after
GPT4 came out.

What Can We Do With This?
• Recommendations? No.
• Data-filling. No, ick.
• Writing stuff, summaries? Yes, ick!
• Classification
• Yes, but not that interesting
• Finding books?!?

Problem 1: Hallucinates ISBNs

Problem 1: Hallucinates Text

Problem 2: Hallucinates Books

Problem 3: Slow
“I haven’t watched this many progress bars
since the 90s.” — Benedict Evans
Problem 4: Out of date
GPT3/4 cut-off at September 2021

Solution: GPT3 + Reality®
• Query GPT3
• Ask for JSON
• Don’t ask for explanations, commentary, etc.
• Parse each book against LibraryThing
• Throw out the crap
• Use only our versions of the data
• Deliver only real books, no comments

Unexpected Problem: JSON hallucinations

Reaction: Good

Learnings
• Hallucinations are a big problem
• Everything is hallucinated, even format
• Reality can put a check on hallucinations
• Don’t use where important
• GPT is still slow
• https://tinyurl.com/LibraryThingAI

Thank you!
Tim Spalding, LibraryThing
March 28, 2023