I uploaded a Persian text today and had Readlang translate some random words in it, just to get a feel how it works with Persian. Everything was more or less as expected.
I then upgraded my subscription to Premium Plus which activated the better AI model (I checked in the settings).
When I checked the text again the most notable difference was that it totally messed up German capitalization (I use German as first language). Before the Persian word for “wall” got translated to “Wand” (correct), but with the better AI model it suddenly was “wand”. Same with other German nouns.
Additionally the “better” AI claimed that the phrase “bottom of the ocean” referred to a given name, which made absolutely no sense. The Persian word for ocean can in theory be a given name, but not in this context. The other AI did not claim anything like that. So it seems that the “better” AI is less intelligent…
I deactivated the better AI model and tested the translations again and now the nouns were correctly capitalized again.
I continued to read in Spanish today (with the better AI model) and also suddenly had the issue with the non-capitalized German nouns. I saved around 500 words before with the smaller model and I never had this issue. This issue doesn’t seem to be specific to Persian apparently.
Sorry about that. I had an issue with the “better” AI being too eager to capitalize words when it wasn’t appropriate which led me to tweak the prompt to tell it to only capitalize if it’s appropriate given its position in the context. For now, I’ve removed this extra instruction on the better AI model when translating into German (for new translations only, if you re-translate the same word in the same context as before it will still give the old cached translation). It’s possible you might now encounter the opposite problem of unwanted capitalization, either way please let me know how you find it after this change.
Note that in my testing, although it does have the capitalization quirk, I’ve found that the larger AI model (gpt-4o) is definitely more aware of obscure Spanish vocabulary than the smaller one (gpt-4o-mini).
(At some point I want to set up a system for collecting feedback on the AI translations and explanations in Readlang over all the different languages and use this to help gradually improve them over time.)
How interesting that the two models behave so differently! And thanks so much for looking into it so fast! At least I’m a little less worried and confused now when the problem only relates to capitalization.
I read a little bit now in Persian (new text, new words) and some (but not all) nouns still weren’t capitalized. There were no occurrences of miscapitalized words though.
I will pay attention to this and report back in a few days.
And that’s great to hear that the better model is also an improvement for Spanish. Ideally I will read some 19th century Spanish in the future so that will come in handy.
So far I have found the explanations extremely helpful, up to the point that they helped me better understand poetry and metaphors which is really awesome! I never got an explanation that was obviously wrong with the smaller model, that’s why I was so confused about the one in Persian.
Sorry for taking so long to report again, but I’m still so confused about this weird behaviour.
I have now translated many more words from Spanish and Persian and I’m not really sure if the changes you made had any effect at all (hard to tell because I only translated a handful of words before).
I still have to correct the capitalization of most of the German nouns. My guess would be that 80% are wrong, although there were times when I felt that it was better, but it was probably just random.
I tried to figure out if there was a pattern as to which nouns were wrong and which were right, but I really have no idea. Seems to be random
But there were still no issues with words that got wrongly capitalized, so it’s not worse than before.
Sorry about this, but I’ve just done some testing and it turns out that the change I made before to the AI prompt when translating into German didn’t actually get deployed . It is now deployed so please give it another try using the better AI model and let me know how it goes.
Haha, no harm done I should have gotten back to you sooner.
It’s a lot better now! I looked up more than a hundred words this evening and all the nouns were correctly capitalized. There was one single instance of a capitalized adjective, but it was a rather obscure one derived from a noun. All other words were fine.
. It is now deployed so please give it another try using the better AI model and let me know how it goes.
I should have gotten back to you sooner.