This is a lofty idea and probably can’t easily be added, but the thought keeps entering my mind as I use the app. I’ve done a quick scan of scholarly literature and it seems that although not without challenges, LLMs can generate and grade reading comprehension tests.
Seeing as I use Readlang primarily to prepare for standardized exams, this would be an amazing feature to have.
I actually tried something like this but it didn’t work reliably enough. The thing I tried was to intersperse multiple choice comprehension challenges throughout the text you are reading. The main problem was that the quality wasn’t consistent enough. Sometimes you multiple of the options would be correct but you would only get graded correct if you happened to pick the right one, and occasionally the question itself seemed to exhibit a misunderstanding of what was actually happening in the text. The other issue was that my idea of putting the questions within a text isn’t for everyone since it interrupts the flow of reading, which is one of the things that Readlang generally tries to avoid. In the end, I iterated on this idea and it turned into the review summaries: Review Summaries - Readlang Blog
If I were to try again I’d now have better LLM models which would surely help (the first time I tried I used the ChatGPT 3.5-turbo model).
Would you like these as part of the current Blitz / Mastery challenges? Or a completely separate challenge type? Or interspersed in the text you’re reading? (No promises about doing this since I’ve got a ton of other things I want to do first, but I’m just curious)
Glad to hear you at least tried it! I imagine this is the sort of task that would benefit both from extensive fine tuning and prompt tuning. Having it as a quiz type was how I imagined it.
