| dfaure | |
| arichardson | |
| apol | |
| thiago |
| Frameworks |
commit 6a18528 introduced escaping of bytes >= 127 to ensure that
KConfig files are valid UTF8.
The simplistic approach with a cutoff results in many escaped bytes
where it is not required. Especially non-western configuration files
would have many escapes.
This commit fixes that by only escaping bytes that are not valid UTF8.
BUG: 403557
FIXED-IN: 5.56
ninja && ninja test
| Automatic diff as part of commit; lint not applicable. |
| Automatic diff as part of commit; unit tests not applicable. |
No objection from me, but I'm no utf-8 expert.
Thiago's input would be very valuable...
| autotests/kconfigtest.cpp | ||
|---|---|---|
| 1774 | What's the purpose of this very short-lived define, compared to just inlining this into the next line? | |
| autotests/kconfigtest.cpp | ||
|---|---|---|
| 1774 | Right, copy-paste leftover. I'll fix it. | |
| src/core/kconfigini.cpp | ||
|---|---|---|
| 683 | This function does operate properly to find valid syntax UTF-8 sequences, but it is neither catching overlong sequences nor UTF-8 content above U+10FFFF (UTF-8 can encode 0x11000 in 4 bytes). See https://code.woboq.org/qt5/qtbase/tests/auto/corelib/codecs/utf8/utf8data.cpp.html#_Z19loadInvalidUtf8Rowsv for potential UTF-8 pitfalls. | |
| src/core/kconfigini.cpp | ||
|---|---|---|
| 683 | A check for U+10FFFF > value is needed. | |
| src/core/kconfigini.cpp | ||
|---|---|---|
| 683 | That's not what an overlong sequence is. You can produce 2-, 3- and 4-byte overlong sequences. See the examples in the Qt test, like 0xc0 0x80. | |
| src/core/kconfigini.cpp | ||
|---|---|---|
| 683 | My understanding of overlong sequences was wrong. I understand now what is meant from your link. | |
Added tests and code fixes to deal with overlong sequences and content
above U+10FFFF.
| src/core/kconfigini.cpp | ||
|---|---|---|
| 683 | Thanks for the unicode explanation. I've added the checks for out of range and overlong now. | |