I’ve just added an experimental feature for premium subscribers allowing you to swap out Google Translate for ChatGPT provided translations. The big advantage is that these translations are aware of the surrounding context. You can enable it from the Settings page here:
They seem to work pretty well in my testing so far. There is slightly more delay compared to Google Translate. Please give them a try and let me know if they work well for you or not!
That’s great, I think it’s a big step up! I tried it out but I didn’t notice that it was slower, it felt the same as Google translate. A much needed feature, I never could trust the Google translation and I end up relying on the explanation feature because it’s so much better.
What happens with the words that are already saved from Google translate? I imagine id need to delete them and start over to have the gpt translation saved.
Great job! 
Edit: After further use I decided it’s better in just about every way. I have noticed no lag/delay and it’s far more accurate. I think it’s a must stay in my opinion. Even if some users experience lag or delay I think it’s worth implementing and trying to smoothen it over time but so far no issues at all.
Awesome!, I’ve just tried it and it works great. Agree with @Marsel_Basholli that it’s better in every way.
I did not notice any delay compared to the business-as-usual. And the translations are indeed much much better.
Only comment / suggestion I would add is:
As a long-standing fan of the browser extension and Android bookmarklet, I would love to also reap the benefits of this, using those very convenient interfaces.
I understand that accommodating the whole explanation in the purposedly minimalist UI of the browser extension may be tricky (although it’d be ever cooler!!!), but if under the wood it can operate based on the context-aware translations, that would be really cool.
Thanks for the feedback! Great to hear that this is working well for you both!
The bookmarklet will now give context-aware translations. I’m working on the Chrome Extension and the new translations should be available there too within the next couple of days.
Hi Steve,
Could you change the prompt to incorporate the Title, Author and Chapter?
Here’s an example: Mark Twain used the word “truck” in Tom Sawyer. Chat gpt translated that as a “vehicle” (see screenshot). Obviously there were no vehicle trucks in the 1840 but the model can’t tell that from the sentence. That is not a limitation of ChatGPT however. If tell it what text I’m reading and I take a picture of the page so it can have more context it gives the correct definition. I’m thinking that if you include the title, author and chapter of the text, you are pointing the model to the actual text and espectially for works in the public domain, the model will know that truck here is not a vehicle. This could be easily accomplished by just changing the words in the prompt.
In this context, “truck” refers to mess or dirt, likely meaning the sticky residue from the jam on the boy’s hands and mouth. The aunt is pointing out the evidence of his mischief, associating it with something undesirable or messy.
Here’s another example where ChatGPT got the wrong meaning. It translated “pumped on our heads” as a physical exertion. Hower, that’s incorrect. This is in the context of skipping school.
Again, it’s not the model’s fault. It’s the lack of context. If I give it more context, it gives this answer:
In this specific context, “pumped on our heads” is an older slang expression meaning “drilled into our minds” or “hammered into our heads” (i.e., forced or incessantly questioned/instructed until it stuck). It conveys the sense that information was pushed or forced upon someone in a persistent way.
Here’s another example where it just can’t give an answer.
There are two asks here,
to change the language of the prompt so that the model is aware from what text this sentence is comming from. That can be acomplied by just adding the title, author and chapter. This is very easy to do I immagine.
expand the context window. This is very helpful for texts that are not in the public domain, texts on which the model wasn’t trained, very recent texts. There are a lot of mistakes like this that the model makes because it lacks context beyond the sentence. And again, it’s not the limitation of the model. If I give it context, it gets it right. It’s how we use these models.
If we have the tools to get correct answer, why not use them? Why learn the incorrect meanings of the words? Is it just to save on compute? I would say just increase the price. I’m not convinced that someone who puts a lot of effort into learning a language is willing to learn the incorrect meanings just to save a few pennies. My problem is that often I don’t know when the meaning is incorrect. I don’t know when I have to pull my phone, take a photo of the page and ask for the definition again and I find myself doing it more than I should, which is frustrating because the advantage of having readlang is so I don’t have to do that. I could read a physical book like this, write a prompt, take photos and type words and get definitions. We all use readlang so we don’t have to do that. Not having the option of extending the context window is just exremely frustrating.
Thank you!
Thanks for the suggestion. I’ve added an experimental feature you can try which will add the title and author to the prompt used by the Explain feature. You can enable it in the Settings page here:
There’s a catch in that these explanations won’t show up in the card edit mode or in practice sessions. Is this important for your use case? Would you be wanting to learn these words in practice sessions?
Note that the above applies only to the Explanations, not the Context-Aware Translations which is what this topic was originally about.
This is great! It looks like the explanations are definitely better. The probability that it will get the right meaning has gone up with this.
But it looks like they’re still not right because it lacks the more immediate context. They are different than the meaning they would have if the model had access to more context. I think this is definitely a big improvement, but I would stil encourage the enlargement of the context since they would clearly make the translation better.
I’m fine with not having them on the flaschards. However, I was refering to both the one word on top and the explantations. Most of the time I read the word translation on the top and that’s still off since it ignores the context. I guess the question is, why not change the prompt for the one word translations as well?
What I’m struggling with is why not try to get the best trasnlations the tools can give us? Why settle for translations that are consistently incorrect when getting the correct ones is possible? This is an instance where some people might say that ChatGPT can’t give the right meanings. Well, yes if we don’t give it the proper context. But why not?
I think we should continue to experiment with these tools until we get them to be as reliable as we possibly can.
Thank you!
It’s mainly due to the extra complexity it adds to the code. Even what I’ve done so far under this feature flag is not good enough for me to release more generally because it would require changes to the practice sessions and the word editor for those new explanations to show up there. What I’ve done here is a quick hack to see how it works.
Regarding the translations, so far I haven’t been reading Cervantes or anything like that, and for the kinds of Spanish texts that I’m reading and that I imagine most language learners will be reading, I think the gpt-4o model is doing an almost perfect job.
I am intrigued by the use case of reading difficult texts in your native language. For these users Readlang has a lot of extra functions that aren’t needed, and a more streamlined experience could be created. Just thinking out loud here though.
Understood on the code side.
Just to clarify, English is my second language and I’m still acquiring vocabulary. The older the text is, the more words I don’t know. I’m struggling with Shakespeare. I’m lost with Chaucer.
The examples I gave are from Tom Sawyer by Mark Twain. The last time I read Mark Twain was in High School when I was in 10th grade. This text is rated as an A2 by Readlang. I don’t know how these grades are calculated but this is what I see in the my library view.
Is an A2 text that is read in the 10th grade an advanced text? Is the expectation that Readlang should give the correct translations of an A2 texts too much to ask? Would most language learners be fine with getting incorrect meanings of an A2 texts?
There are multiple layers to this and I’ve talked about many different ideas. Reading texts with AI, using advanced models, etc. In this particular case, I’m just pointing out that the current translations of an A2 text that is read in 10th grade are incorrect. If an English teacher tries out the text she assigned in class on Readlang, would she recommend it to her students?
Part of the reason I’m reading in English with Readlang is to expand my vocabulary, but also to test it, to see how it works. It’s easier for me to figure out whether the translation works well in English than in Spanish or German. So I’m pointing things out in English because I know it the most.
If I use it in German, can I read Goethe? Can I trust it? Or can I read Mozart’s Zeuberflote libretto? That text is over 200 years old and I was assigned to read it in my 2nd year of German. Would my German professor be comfortable with the translation I see on Readlang. Again, this is after 1 year of studying German, so still a beginner stage.
The bar I’m talking about in this particular case is very low. It’s about translating beginner texts. Is getting a large % of those words wrong acceptable to most language learners?
Again, I’m a big fan and believer in this project. The interface is great, the concept is great, the LLM models work if use right, but the error rate on an A2 text is just too high. I’m talking about the basics here. The other things I talked about in the other threads will happen, whether we do them here or elsewhere, but in this thread I’m just talking about the basics, getting the correct meaning in context.
Here’s why this is dangerous. If I look a word in the dictionary, and I see 5 definitions, I’m aware that there are 5 definitions and I’ll make a judgement on which one is most suited. But if I believe that the model will pick the correct definition for me in this context and I’m trusting the model, then I will just assimilate that definition. And a certain percentago of those words will be incorrect. My knowledge of the language will be faulty. And I’ll have to spend significantly more effort ulearning the wrong definitions and learning the correct ones.
Again, if we can’t trust the translations of an A2 text, it undermines the entire project.
Honestly, I don’t think this is doable without enlarging the context. When we read, we have the larger context. When we look at a word in the dictionary and see 5 or 10 meanings, we have the larger context in our head. Why would we expect that the models could pick the right definition without the context? We can go further, I don’t think people can pick up the correct definitions based just on a small sentence in many cases. Why would we ask the model to do something that a person couldn’t do?
What I’m doing here is challenging the assumption that a one sentence context is enough. I would argue that it’s not. The LLM models can’t do it. People can’t do it. It works in most cases, but is most cases good enough. If I’m teaching you a language and 5% of the words I’m teaching are incorrect, is that an acceptable threshold? (I don’t know if it’s 5% or 2%, but I’d like that to be close to 0).
Since we’re trying to figure these things out, could we expand the context for the experimental feature if that’s easier? We can play with that for a while, figure out how large the context should be and see how it works. If we can get that section to work properly, then I can at least rely on that until we figure a better way for the rest of the features.
Thank you!
This is another assumption I would challenge. Are we sure about that? I think that with a one sentence context window the model would struggle with a lot of texts that would be assigned by a professor in a 2nd year language course. Many beginner texts have short sentences. It’s often very hard to tell the correct meaning in a short sentence.
