Status: I built it and shut it down after — here’s a demo video
https://www.loom.com/embed/773f3e141a7a441fa54aecd91524f3a3
The fastest and most fun way for me to learn is to shoot a whole bunch of questions. That's why I love office hours. Imagine being able to show up to an expert's office, and ask any question. I am so grateful that most of the people discovering things on the cutting edge are also educators.
I don't think a structured, sequential syllabus is not the best way to learn. Motivation to learn: curiosity, comes from asking questions. Maybe I just want to learn how the COVID-19 vaccine works — I don't need to study a whole introductory biology course to learn about this. But maybe I run into the fact that the best vaccines are mRNA based, and then learn about what mRNA is, then the central dogma. All my learning started with a question I had — not one that was presented to me. I was in charge of my learning, not a syllabus.
It's pretty well established that the average student learns much better when they have a personal tutor, that they can ask questions to.
The average student tutored one-to-one using mastery learning techniques performed two standard deviations better than students educated in a classroom environment with one teacher to 30 students
Historically, this has been prohibitively expensive to do. Hiring an expert for every student gets very costly very quickly. Students are left to watch lectures and fend for themselves.
They're still curious though! They'll google questions they have — sometimes not getting anywhere close, because they just don't know how to phrase questions. Or maybe, the answers are just plain incomprehensible.
The next step may be asking a question online on Quora or Stack overflow. This means waiting several days before you get an answer, and can continue discourse. The feedback loops from question -> reply is incredibly slow.
Today though, I think we might have a real shot to get everyone a personal tutor. Large language models like GPT-3 are getting really good. These are big neural nets (175B+ parameters), trained on a corpus of text representing a significant percentage of the internet. They're trained to essentially autocomplete text.
The prototype on riff.quest lets you ask questions like you're in office hours. It combines several LLM completions along with querying the internet on each message to fetch relevant information through semantic search, and then generates a relevant reply.
There's also promising progress on understanding the accuracy of LLMs. That means this AI tutor will be able to actually know when they're wrong!