Skip to content

Update dotstrings parser to handle entries with multi-line values - #12

Open
Salman (fla-t) wants to merge 4 commits into
microsoft:masterfrom
fla-t:flat/support-for-multiline-strings
Open

Update dotstrings parser to handle entries with multi-line values#12
Salman (fla-t) wants to merge 4 commits into
microsoft:masterfrom
fla-t:flat/support-for-multiline-strings

Conversation

@fla-t

Copy link
Copy Markdown
Contributor

The dotstrings parser has been updated to handle multi-line entries in the format

"key" = "value
that spans
multiple lines"; 

This allows for more flexibility in writing and parsing dotstrings files

The dotstrings parser has been updated to handle multi-line entries in the format "key = value;". This allows for more flexibility in writing and parsing dotstrings files.
Comment threaddotstrings/parser.py
comment = re.compile(r"(\'(?:[^\'\\]|\\[\s\S])*\')|//.*|/\*(?:[^*]|\*(?!/))*\*/", re.MULTILINE)
whitespace = re.compile(r"\s*", re.MULTILINE)
entry = re.compile(r'"(.*)"\s*=\s*"(.*)";')
entry = re.compile(r'"([^"]*?)"\s*=\s*"((?:[^";]|"(?!\s*;))*?)";', re.DOTALL)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately this will break on strings that contain quotes. e.g. NSLocalizedString("Hello \"World\"", "Some Comment") gets turned into "Hello \"World\"" = "Hello \"World\"";.

I think the second change on this line will also stop the string containing ;.

@fla-tSalman (fla-t)Jun 28, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first change is there so that the first match group doesn't end up matching the whole string (even matching the "value" part).

But yeah it wouldn't work when there are more than two quotes, in the "key" part of the string (the two quotes around the key itself). This was intentional because I didn't thought there would be any "key" that would have more quotes than two.

If we really want to cater keys that have quotes inside of them, we can replicate the regex that we have for value part, but without the semicolon

entry = re.compile(r'"((?:[^";]|"(?!\s*;))*?)"\s*=\s*"((?:[^";]|"(?!\s*;))*?)";')

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The second change is finding a ";" to match the value part of the string.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change the test file I have here to contain edge cases?

Comment threadtests/test_simple.py
Comment threadtests/test_simple.py Outdated
@fla-t

Salman (fla-t) commented Jun 28, 2024

Copy link
Copy Markdown
ContributorAuthor

Also there is a line that I hate everytime I take a look at it.

comment = comment[::-1].replace("/*", "", 1)[::-1]

# can easily be this
comment = comment.removeprefix("/*")
comment = comment.removesuffix("*/")

Or maybe I am missing something? :)
(I would love if we can add this change in this PR)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@fla-t@dalemyers