Repository navigation
Fix patchedast aborting on ''' inside a double-quoted f-string - #844
marlon-costa-dc wants to merge 9 commits into
Conversation
end_quote_char() picked the closing delimiter by max(len(quote)) over every
quote char found anywhere in the f-string body, so a ''' sequence inside a
double-quoted f-string (e.g. f"cfg = '''\n{value}") was selected as the
delimiter. That produced a reconstructed token consume_joined_string() could
not locate, raising `ValueError: substring not found` -> MismatchedTokenError,
which aborts rename/inline for the entire module.
Pick the delimiter the f-string literally ends with (QUOTE_CHARS is ordered
longest-first, so """/''' win over "/' correctly) so the closing quote matches
the opening one regardless of quote characters appearing in the body.
Adds a regression test; full patchedast, rename and inline suites stay green.
There was a problem hiding this comment.
Pull request overview
Fixes patchedast f-string reconstruction when the f-string body contains quote sequences (notably ''') that previously confused _JoinedStr.end_quote_char(), causing consume_joined_string() failures and aborting refactorings for the whole module.
Changes:
- Update
_JoinedStr.end_quote_char()to select the closing delimiter based on what the source literally ends with (rather than the longest quote found in the body). - Add a regression test covering a double-quoted f-string containing
'''inside its body. - Document the fix in
CHANGELOG.md.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
rope/refactor/patchedast.py |
Fixes f-string closing delimiter detection to avoid mismatches caused by quote sequences in the f-string body. |
ropetest/refactor/patchedasttest.py |
Adds a regression test proving the bug and preventing future regressions. |
CHANGELOG.md |
Notes the bug fix and its impact on refactorings. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
# Conflicts: # CHANGELOG.md
Addresses review: end_quote_char fell back to the last character when no quote delimiter matched, contradicting its intent and masking parsing bugs. It now raises MismatchedTokenError naming the line, like the other token-matching failures in patchedast.
|
Rebased this fix onto current master (post-#896): the old The updated implementation reads the delimiter from whatever quote sequence actually terminates the region and raises |
# Conflicts: # CHANGELOG.md
|
Merged current master into this branch; the changelog entry now sits under the upcoming release. The bug is still present on master. With the regression test's source, |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #844 +/- ##
=======================================
Coverage 95.44% 95.45%
=======================================
Files 134 134
Lines 26910 26924 +14
=======================================
+ Hits 25685 25699 +14
Misses 1225 1225 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Our open python-rope/rope PRs carry tests that the fork release absorbed in behaviour but not in tests. Each passes against the current code: - python-rope#908 test_assignment_hints_ignore_names_that_are_not_assigned - python-rope#909 test_scope_region_rejects_decorator_without_source_positions, test_scope_region_rejects_decorator_without_source_marker - python-rope#920 test_hint_param_on_a_hierarchy_python_refuses - python-rope#910 test_find_module_skips_python_path_files - python-rope#844 test_format_string_region_without_closing_quote_is_rejected Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
Description
patchedast._JoinedStr.end_quote_char()selects the closing delimiter bymax(len(quote))over every quote character found anywhere in the f-string body. When a'''sequence appears inside a double-quoted f-string, e.g.:f"cfg = '''\n{value}"the
'''in the body is picked as the delimiter instead of the actual closing". That produces a reconstructed token thatconsume_joined_string()cannot locate, raising:which surfaces as
MismatchedTokenErrorand aborts rename / inline for the entire module — a single such f-string anywhere in a project breaks refactoring of that file.Root cause
end_quote_char()chose the quote by length precedence rather than by which delimiter the f-string literally ends with. The opening quote (start_quote_char) was already correct; the closing detection was not symmetric.Fix
Pick the delimiter the f-string literally ends with.
QUOTE_CHARSis ordered longest-first (""",''',",'), so triple-quoted strings are still matched before single-character quotes, and the closing quote now always matches the opening one regardless of quote characters in the body.Reproducer (before the fix)
Testing
test_handling_format_strings_with_triple_quote_in_bodyinropetest/refactor/patchedasttest.py(red before, green after).ropetest/refactor/= 944 passed, 7 skipped, 1 xfailed.patchedasttest.py= 141 passed, 6 skipped.Checklist