I'd support such a change for the BackgroundChecker as well. KTextEditor needs a way to do its own language detection or override the detected language.

In D26342#585867, @mludwig wrote:

I'd support such a change for the BackgroundChecker as well. KTextEditor needs a way to do its own language detection or override the detected language.

I've updated the diff

@mludwig: ok with the current patch?

Can we get some traction on this?
It's a relatively light change that doesn't affect the default Sonnet behavior.

without having used Sonnet much, this seems the wrong API to me.

Are you saying that it can happen that you tell Sonnet "use this language" and it goes and say "nah i'll ignore you and do my thing".

It seems to me that what would make sense is that the "use this language" is what sets autoDetectLanguageDisabled to false, or at least the function that should get a new overload saying setLanguage(¿qstring? language, bool disableAutoDetection) if we want to be sure not to change existing behaviour

In D26342#619785, @aacid wrote:

without having used Sonnet much, this seems the wrong API to me.

Are you saying that it can happen that you tell Sonnet "use this language" and it goes and say "nah i'll ignore you and do my thing".

Yep, that's right, and it won't even tell you which language it has detected. When the autodetection feature was introduced, it fundamentally changed the behaviour of Sonnet, breaking KatePart's spell checking in the process.

It seems to me that what would make sense is that the "use this language" is what sets autoDetectLanguageDisabled to false, or at least the function that should get a new overload saying setLanguage(¿qstring? language, bool disableAutoDetection) if we want to be sure not to change existing behaviour

Ideally, the autodetection feature should be redesigned as it is only usable for simple text fields in its current form. KatePart requires a more sophisticated way of detecting languages(s).

@Simon: If you add a call 'setAutoDetectLanguageDisabled(true)' to 'BackgroundChecker::changeLanguage' and 'BackgroundChecker::setSpeller', you should be able to take Albert's suggestion into account.

For the Highlighter it should be similar.

I don't see a use case where you would want to set the Language, but then let the system override it (and only if the user has auto-detection).
I left the option in the API to "setAutoDetectLanguageDisabled" to false, for instance if you are not sure which language is being used anymore and want to let the system re-decide by itself.

Would it make sense to enshrine this behaviour with an autotest?

In D26342#623729, @aacid wrote:

Would it make sense to enshrine this behaviour with an autotest?

Sorry what do you mean by that? Similar to tests/BackgroundTest.cpp ?

In D26342#632842, @sdepiets wrote:

In D26342#623729, @aacid wrote:

Would it make sense to enshrine this behaviour with an autotest?

Sorry what do you mean by that? Similar to tests/BackgroundTest.cpp ?

No, something like the stuff autotests/

one example would be something like, use a text written in french that the autodetect algorithm properly detects as french, but then you go and force say German on it and then you go and ask for which language is and it should say German when autodetetaLanguageDisabled = true but then if you go and say setAutoDetectLanguageDisabled(false) again it should say again French

Am i making sense?

You need to update the \since markers?

After that i guess you can commit unless anyone really disagrees?

@mludwig @cullmann ?

This actually breaks language auto-detection for me in the KMail composer.

Testcase:

• New Mail
• I type "Bonjour," in the body

Before:
It's detected as French and not underlined as a typo

After:
The language remains English, and the word is underlined as a typo

Workaround:
Tools / Spelling / change language from English to French

Too bad I'm realizing this is the reason for the regression only today (day of 5.71 release) by looking at the KF-5.71 changelog :(

In D26342#675142, @dfaure wrote:

This actually breaks language auto-detection for me in the KMail composer.

Testcase:

• New Mail
• I type "Bonjour," in the body
  
  Before: It's detected as French and not underlined as a typo
  
  After: The language remains English, and the word is underlined as a typo
  
  Workaround: Tools / Spelling / change language from English to French
  
  Too bad I'm realizing this is the reason for the regression only today (day of 5.71 release) by looking at the KF-5.71 changelog :(

I don't think that's a regression, in the previous behavior you could try to set any language to proofread, it would always auto-detect "Bonjour" as French, thus the "Tools / Spelling / change language" had not effect if autodetect was enabled at system level (while autodetection should be an application or even case by case decision).

It may not seem like an issue for simple cases, but actually for mixed contents (i.e. an email that is 50% French, 50% English) that would be detected as English, you would have no way at all to check the French text without disabling system-wide autodectection.

Calling setAutoDetectLanguageDisabled(false) restores the previous behavior

In D26342#675164, @sdepiets wrote:

In D26342#675142, @dfaure wrote:

This actually breaks language auto-detection for me in the KMail composer.

Testcase:

• New Mail
• I type "Bonjour," in the body
  
  Before: It's detected as French and not underlined as a typo
  
  After: The language remains English, and the word is underlined as a typo
  
  Workaround: Tools / Spelling / change language from English to French
  
  Too bad I'm realizing this is the reason for the regression only today (day of 5.71 release) by looking at the KF-5.71 changelog :(

I don't think that's a regression, in the previous behavior you could try to set any language to proofread, it would always auto-detect "Bonjour" as French, thus the "Tools / Spelling / change language" had not effect if autodetect was enabled at system level (while autodetection should be an application or even case by case decision).

It may not seem like an issue for simple cases, but actually for mixed contents (i.e. an email that is 50% French, 50% English) that would be detected as English, you would have no way at all to check the French text without disabling system-wide autodectection.

Calling setAutoDetectLanguageDisabled(false) restores the previous behavior

The problem here is that David probably never set any language on purpose.

you could argue that this is a deficiency of the UI, should have a checkbox for "enable spellchecking" and another for "enable spellchecking *exactly* in this language"

And this is were we failed, we changed the default behaviour and that's probably not the greatest of the ideas in retrospect. Even if we could not see why anyone would set a language and would still want the auto language detection to be enabled, well it seems that at least David wanted because it's been the behaviour for ages :D

Not sure what's the correct path forward though :/

In D26342#675180, @aacid wrote:

In D26342#675164, @sdepiets wrote:

I don't think that's a regression, in the previous behavior you could try to set any language to proofread, it would always auto-detect "Bonjour" as French, thus the "Tools / Spelling / change language" had not effect if autodetect was enabled at system level (while autodetection should be an application or even case by case decision).

Autodetection *is* an application setting, it's in Settings / Spellchecker... / [x] Enable autodetection of language

It may not seem like an issue for simple cases, but actually for mixed contents (i.e. an email that is 50% French, 50% English) that would be detected as English, you would have no way at all to check the French text without disabling system-wide autodectection.

In my experience a mixed email was perfectly handled, with each paragraph having a different language.
See http://www.davidfaure.fr/2020/Screenshot_20200613_194827.png

But I think you're talking about the spellchecking dialog (which I never use, except this month because of the need to find a workaround).
My concern is for autodetection and background spellchecking. If a fix for the spellcheck dialog introduces a change of behaviour for the background spellchecking, that's a regression.

Calling setAutoDetectLanguageDisabled(false) restores the previous behavior

Changes in KF5 should NOT require applications to adapt in order to restore previous *working* behaviour.

The problem here is that David probably never set any language on purpose.

Right.
In fact, if I open the settings dialog, it has "français" checked in the top listview, and only that one.
Meanwhile the Tools / Spelling dialog shows "American English (UnitedStates)" in the combobox.
This particular issue happens even with this patch reverted. Something's rather broken.

you could argue that this is a deficiency of the UI, should have a checkbox for "enable spellchecking" and another for "enable spellchecking *exactly* in this language"

Not sure I understand this.

And this is were we failed, we changed the default behaviour and that's probably not the greatest of the ideas in retrospect. Even if we could not see why anyone would set a language and would still want the auto language detection to be enabled, well it seems that at least David wanted because it's been the behaviour for ages :D

*I* don't really set a language. Maybe KMail does though. @mlaurent might know more?

Please see below proposal to restore the previous default behavior

https://invent.kde.org/frameworks/sonnet/-/merge_requests/1