Repository navigation
patchedast: locate multi-line f-strings, replacement fields and match-sequence delimiters by parser coordinates - #882
Conversation
A PEP 701 replacement field may span lines, which the textual string pattern cannot match, so JoinedStr raised MismatchedTokenError; its region now comes from the node's own coordinates, implicit concatenation included. A MatchSequence took every character before its first pattern as its opening delimiter, so a grouped first pattern (`((a | b), c)`) produced `((`; the delimiters are now the bracket pair that encloses the whole node, found by tokenizing its segment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In an implicit concatenation mixing f-strings and plain strings, the
field's opening brace was searched for as text and matched a "{" inside
a plain part (`"({})," f"{a}"`), shifting the field's region and
duplicating source on write. Since Python 3.12 the parser locates each
FormattedValue at its own "{"; the walker now starts there when the
coordinate points at one. test_extract_method_f_string_false_format_value
_in_regular_string, marked as an expected failure for this bug, passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds the parser-coordinate patchedast fixes (upstream PR python-rope#882): multi-line f-string replacement fields, replacement-field braces in plain parts of a concatenation, and grouped first patterns in match sequences. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Reworked the residual of this PR on top of current master (post-#896): the f-string bounding via parser coordinates landed through #896's Available for comparison at |
|
The patched-AST cleanup in #896 overlaps with the f-string parts of this PR, but the MatchSequence delimiter work still appears useful. Could you update this PR against current master and narrow it to the remaining MatchSequence behavior? Please retain regression tests for grouped first patterns, such as The Once the remaining change is isolated, a fresh CI matrix should give us much more useful results. |
The f-string coordinate work of this pull request landed through python-rope#896, so the merge takes master's patchedast, its tests and the changelog, and narrows the pull request to the remaining MatchSequence delimiter fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_get_surrounding_parens stripped the text around the first and last
patterns and emitted a please-report RuntimeWarning whenever that
text was not a single bracket, which happens for grouping parentheses
of an element (case (("a" | b), c)) and for sequences whose elements
are separately parenthesized (case (1), (2)). Tokenize the segment
instead: the sequence has its own delimiters exactly when the bracket
opening its first character closes at its last character.
(cherry picked from commit 431061b)
The multiline replacement field case already passes on master after python-rope#896. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Updated against current master and narrowed to the MatchSequence delimiter behaviour, as requested:
Tested locally on Python 3.10, 3.11, 3.13 and 3.14. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #882 +/- ##
==========================================
+ Coverage 95.44% 95.48% +0.03%
==========================================
Files 134 134
Lines 26910 26936 +26
==========================================
+ Hits 25685 25719 +34
+ Misses 1225 1217 -8 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
mcepl
left a comment
There was a problem hiding this comment.
The narrowed MatchSequence change looks good to me. The regression tests cover the grouped-pattern and separately parenthesized-element cases, and the checks pass. Please update the PR description to remove the f-string fixes, which are no longer part of this diff and landed via #896.
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
Description
Three
patchedastdefects found by walking a ~6,300-file Python 3.13 codebase withget_patched_ast(source, True)and requiringwrite_ast()to reproduce each file exactly:x = (f"{a}{\n b\n}{c}").upper()raisedMismatchedTokenError:_JoinedStrlocated the literal with the textual string pattern, whose short-string form excludes newlines. The literal's region now comes from the node's own coordinates (ast_adapter), implicit concatenation included."({})," f"{a}"the field's{was searched for as text and matched the{inside the plain part, so theFormattedValueregion started at(andwrite_astduplicated source. Since Python 3.12 the parser locates eachFormattedValueat its own{; the walker starts there when the coordinate points at one (older parsers give the enclosing literal's position, which is left untouched). This also fixes the case covered bytest_extract_method_f_string_false_format_value_in_regular_string, whose@expectedFailureis removed.match-casestatement syntaxes #623).case (("a" | None), None):warnedUnexpected character in MatchSequence's opening_paren <((>and duplicated the source on write, because every character before the first pattern was taken as the delimiter. The sequence's delimiters are now the bracket pair that encloses the whole node, found by tokenizing its segment;case (1), (2):, whose outer parentheses belong to its elements, is also handled.Checklist
ropetest/refactor/patchedasttest.pyfail on currentmasterand pass here; the previously expected-failure extract test passes.🤖 Generated with Claude Code