Use new character width code based on Unicode 11

Authored by mglb on Oct 3 2018, 3:06 PM.

Description

Use new character width code based on Unicode 11

Summary:
Adds a code for getting character width togeter with LUTs generated
using uni2characterwidth from Unicode 11 lists.

Skin tone, flags, gender, and other emoji with and modifer are not
joined (you will see e.g. a skin tone square + generic yellow emoji).
I think joining them would cause problems in most editors, command line
prompts, and other programs which use character width data, as the
characters would behave as combining or emoji depending on context (like
ligatures).

Examples:

  • light thumb up: 👍đŸģ
  • dark thumb up: 👍đŸŋ
  • Polish flag: đŸ‡ĩ🇱

This behavior is allowed:

It is possible to add support for sequences, but those would work
only for a string width functions.

Some characters which can be presented as emoji are narrow (e.g. ✖ī¸, Šī¸).
Those characters are listed without "presentation" mode, which means
they should be rendered as text by default (real presentation depends on
renderer and/or font). Noto Sans Color Emoji renders them as wide,
DejaVu Sans as narrow. Vim, bash and zsh treat them as narrow, so I made
them narrow.

https://unicode.org/reports/tr51/#Presentation_Style

BUG: 396435
BUG: 378124
BUG: 392171
BUG: 339439

FIXED-IN: 18.12

Depends on D15757

Test Plan:

  • Look at emoji_test.txt - emojis should look "normal" (two characters

width).

  • Look at GLASS.txt - characters width should look correct.
  • CharacterWidthTest should pass.
  • perl -XCSDL -e 'print map{chr($_), " "} 1..0xffff'

Reviewers: Konsole, VDG, hindenburg

Reviewed By: Konsole, hindenburg

Subscribers: hindenburg, broulik, ngraham, konsole-devel

Tags: Konsole

Differential Revision: https://phabricator.kde.org/D15758

Details

Committed
hindenburgOct 3 2018, 3:11 PM
Reviewer
Konsole
Differential Revision
D15758: Use new character width code based on Unicode 11
Parents
R319:bfb91aac49f2: Check clipboard->mimeData() is valid
Branches
Unknown
Tags
Unknown