Change internal character type size to 32 bit

Authored by mglb on May 16 2018, 1:49 PM.

Description

Change internal character type size to 32 bit

Summary:
Currently Unicode uses 0x10FFFF code points. It is possible to represent
all of them with up to two 16 bit values (UTF-16), but this makes it
harder to e.g. check for their width.

Some test cases were changed. Originally they were added with an
assumption that the code point will be truncated to 16 bit value and
as a result changed to another code point.

Test Plan:

  • All code points <= 0xFFFF should work as before
    • Start the same tmux session in two Konsoles
    • Change background to fully transparent in one of them and put it in front of the other one, so that all text will overlap
    • Generate characters with (you can pipe it to fold -s | less -r):
perl -XCSDL -e 'print map{chr($_), " "} 1..0xffff'
  • Compare output visually.
  • Code points > 0xFFFF should not be truncated to 16 bits
    • "𝐀" and "퐀" should be different characters
  • Some code points > 0xFFFF should have single width
    • Vertical lines below should align:
|𝐀 |
|𠀋|

Reviewers: Konsole, hindenburg

Reviewed By: Konsole, hindenburg

Subscribers: konsole-devel, hindenburg, Konsole

Tags: Konsole

Differential Revision: https://phabricator.kde.org/D12236