Skip to content

bpo-23894: Make rb'' strings work in lib2to3 - #1724

Merged
ambv merged 1 commit into
python:masterfrom
ambv:bpo-23894-rb
May 22, 2017
Merged

bpo-23894: Make rb'' strings work in lib2to3#1724
ambv merged 1 commit into
python:masterfrom
ambv:bpo-23894-rb

Conversation

@ambv

@ambvambv commented May 22, 2017

Copy link
Copy Markdown
Contributor

This partially solves bpo-23894. A separate PR coming later to support f-strings.

@ambvambv added type-bug An unexpected behavior, bug, or error needs backport to 3.6 labels May 22, 2017
@ambv
ambv requested a review from ned-deilyMay 22, 2017 18:46
@ambvambv changed the title Make rb'' strings work in lib2to3bpo-23894: Make rb'' strings work in lib2to3May 22, 2017
@ambv
ambvforce-pushed the bpo-23894-rb branch 2 times, most recently from 76b0e9d to d330004CompareMay 22, 2017 19:04
ambv
ambv commented May 22, 2017
View reviewed changes
Comment threadLib/lib2to3/pgen2/tokenize.py Outdated

@ambvambvMay 22, 2017

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"String" didn't previously include b but this is suspicious to me. Why wouldn't it? That way b'' literals were only PseudoTokens (via ContStr), not actual Tokens (via String).

@gvanrossumgvanrossumMay 22, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG

ned-deily
ned-deily approved these changes May 22, 2017
View reviewed changes

@ned-deilyned-deily left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the test: LGTM. As far as the tokenizer changes, I'm not familiar enough with lib2to3 to provide a useful review. Assuming other reviewers have no objections, I'm fine for this going into 3.6.2rc.

@ambv

ambv commented May 22, 2017•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Tests are failing because I used an f-string in the test and lib2to3 is parsing that test file as part of fixers. f-strings are not supported yet in lib2to3. Switching to using a classic str.format fixes it. I'm updating the PR now.

UPDATE: as expected, tests pass now.

Lukasz Langa
Make rb'' strings work in lib2to3
006c88d
This partially solves bpo-23894.
@ambv
ambv requested review from gvanrossum and vstinnerMay 22, 2017 19:42
@ambv
ambv merged commit 0c4aca5 into python:masterMay 22, 2017
ambv added a commit to ambv/cpython that referenced this pull request May 22, 2017
@ambvLukasz Langa
[3.6] Make rb'' strings work in lib2to3 (pythonGH-1724)
b2c5e99
This partially solves bpo-23894.
(cherry picked from commit 0c4aca5)
ambv added a commit that referenced this pull request May 22, 2017
@ambv
[3.6] Make rb'' strings work in lib2to3 (GH-1724) (#1730)
1398b1b
This partially solves bpo-23894.
(cherry picked from commit 0c4aca5)
@vstinner

vstinner commented May 23, 2017

Copy link
Copy Markdown
Member

I was requested for a review, but after I completed my review, I see that the change was already merged :-D It's fine, don't worry. I just want to give my post-commit LGTM! Nice change. Thanks @ambv for taking care of 2to3 ;-)

@jstasiakjstasiak mentioned this pull request Jul 17, 2017
Closed
@ambv
ambv deleted the bpo-23894-rb branch July 12, 2021 11:22
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ned-deilyned-deilyned-deily approved these changes
@vstinnervstinnerAwaiting requested review from vstinner
@gvanrossumgvanrossumAwaiting requested review from gvanrossum

Assignees

No one assigned

Labels

type-bugAn unexpected behavior, bug, or error

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@ambv@vstinner@gvanrossum@ned-deily@Mariatta@the-knights-who-say-ni