Support for underscores in numeric literals (Python 3.6)
ClosedPublic

Authored by januz on Oct 20 2017, 12:20 AM.

Details

Summary

This patch adds support for underscores in numeric literals for Python 3.6 (from PEP 515)

BUG: 385422

Test Plan

Test different numbers with underscore in the code editor sample.

an_int = 100_000_000
a_float = 100_0000_000.0
some_hex = 0xEAD2_DF24_3D

Diff Detail

Repository
R216 Syntax Highlighting
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
januz created this revision.Oct 20 2017, 12:20 AM
Restricted Application added a project: Frameworks. · View Herald TranscriptOct 20 2017, 12:20 AM
Restricted Application added a subscriber: Frameworks. · View Herald Transcript
cfeck edited the summary of this revision. (Show Details)Oct 20 2017, 12:23 AM
cfeck added a reviewer: Kate.

Hi,
I assume the regex are right ;=)

For the patch:
a) I think the version number must be increased
b) an extended unit test would be nice, to really show that the regex are right.

januz added a comment.Dec 9 2017, 11:10 PM

b) an extended unit test would be nice, to really show that the regex are right.

Hi, sure. Is there any documentation on how to make those tests? I noticed there are some ref and input files, do I just modify those and run the tests?

In D8371#178009, @januz wrote:

b) an extended unit test would be nice, to really show that the regex are right.

Hi, sure. Is there any documentation on how to make those tests? I noticed there are some ref and input files, do I just modify those and run the tests?

Yes, in the end that is the right process.
Add your new fragment to them, run them, look at the diffs, if they are OK, update the ref.

Btw: To update the reference, you can simply execute

build/frameworks/syntax-highlighting/autotests/update-reference-data.sh

Note, this script is in your *build* folder (not the git checkout folder).

januz updated this revision to Diff 24317.Dec 23 2017, 12:03 AM
  • Tweaked regexes to fit better with the other regex in the file
  • Version bump
  • Added autotests and updated references

Thanks for the tips guys!

This patch still has two problems (that I'm aware :) ):

  • I can't get it to match the dot in the float regex. The regex itself is ok (https://regexr.com/3idtj), but it just doesn't work.
  • The regexes don't work inside lists or dictionaries. I've noticed highlighting for complex numbers doesn't work either though, so it might not be related to this patch.
dhaumann accepted this revision.Feb 1 2018, 7:48 PM

Looks good. If you have more updates, please let us know by sending more patches :-)

This revision is now accepted and ready to land.Feb 1 2018, 7:48 PM
This revision was automatically updated to reflect the committed changes.