Repository navigation
PR triage: compact bullet comment; judge again on every ready_for_review - #5308
Conversation
- dev-tools/scripts/pr-triage-llm.py: render a bullet per judgement instead of a table. An agreeing row states what the PR has (included, not needed, or the changelog type) and keeps the reason in an HTML comment. A mismatch row shows the suggestion and what the PR has, the reason, and the confidence only when it is below high. Unanswered rows say "Unclear; please assess.". The assessed commit and the key legend move into or out of the comment. - .github/workflows/pr-triage-llm.yml: drop the "already triaged" check and its permission; a PR that returns to ready is judged again. - .github/pr-triage/judgements.json: choice_label names the changelog type in the comment (documented in _doc). - dev-tools/scripts/tests/test_pr_triage.py: cover the new rendering. Generated-by: Claude Sonnet 5.5
|
🤖 Example of the new format: a real run with Automated PR readiness judgements (LLM-generated; may be wrong; not updated on later pushes). Consider enabling Allow edits from maintainers so Solr maintainers can contribute to this branch (optional). 2 possible mismatch(es) between what this change warrants and what the PR includes.
Markdown source**Automated PR readiness judgements** (LLM-generated; may be wrong; not updated on later pushes).
Consider enabling [Allow edits from maintainers](https://docs.github.com/en/pull-requests/how-tos/work-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork) so Solr maintainers can contribute to this branch (optional).
**2 possible mismatch(es)** between what this change warrants and what the PR includes.
- ✅ [Jira issue in title](https://github.com/apache/solr/blob/main/CONTRIBUTING.md): already present
- ⚠️ [Changelog](https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc#_changelog_types_reference) **— differs from the entry** (suggest type: fixed; PR has type: changed): The change fixes a user-visible issue where INSTALLSHARD operations would log noise or fail gracefully due to the presence of 'write.lock' files in offline index directories, which is a common scenario for users copying live indices.
- ✅ [Reference Guide update](https://github.com/apache/solr/tree/main/solr/solr-ref-guide): not needed <!-- This is an internal robustness fix that handles a specific file type gracefully; it does not introduce new configuration, parameters, or user-facing behaviors that require documentation in the Reference Guide. -->
- ✅ [Upgrade notes](https://github.com/apache/solr/tree/main/solr/solr-ref-guide/modules/upgrade-notes/pages): not needed <!-- The change is additive and backward-compatible, simply skipping a file that was previously causing issues, without removing or renaming any APIs or configuration. -->
- ⚠️ Tests **— missing**: The PR modifies production logic in RestoreCore.java to skip write.lock files, and the diff includes new tests in AbstractInstallShardTest.java to verify this behavior.
<!-- Assessed commit f220b849b9008f46a5c659703f58e08cae13b0b3 -->🤖 Generated with Claude Code |
|
Automated PR triage (LLM-generated; may be wrong). Checked once, when the PR was opened or marked ready; not updated on later pushes. 1 possible mismatch(es) between what this change warrants and what the PR includes.
|
|
I did quite a few iterations perfecting the language / presentation, and I'm now really happy with it. Perhaps the pre-bullet header sentence can be dropped, e.g. the "2 possible mismatch(es) between what this change warrants and what the PR includes." since IMO the true first line header is fine and the bullet information reads well enough. |
janhoy
left a comment
There was a problem hiding this comment.
I too thought the table was a bit of a monster, this is a nice improvement. Over in the solr-operator port I also did a collapse to one single row if all checks were green...
Generated-by: Claude Sonnet 5.5
The comment is now a bullet per judgement instead of a table. A row that
agrees with the PR says what the PR has (included, not needed, the
changelog kind, or "no concern" for the flag judgements that have nothing
to compare with) and keeps the model's reason in an HTML comment, visible
only in the Markdown source. A row that disagrees shows what the model
suggests and what the PR has ("suggest added; PR has fixed"), the
confidence when it is below high, and the reason. Unanswered rows read
"Unclear; please assess." The headline, unanswered count and footer
legend are gone, the header reads "Automated PR readiness judgements",
and the assessed commit is a hidden comment. The one-line collapse when
every judgement agrees is kept.
The workflow no longer looks for an earlier triage comment before
judging, so a PR toggled draft -> ready more than once is judged again
and gets a fresh comment. The comment marker and the issues:read
permission go with it; the API key check stays.
Solr's choice_label key is not ported; the changelog kind is shown bare.
A choice judgement can name the value it is about, so the changelog row reads "kind: added" and "suggest kind: fixed; PR has kind: changed" instead of a bare word that means nothing on its own. Both changelog judgements set it to "kind", the field name in Chart.yaml. The label is optional and is left off an agreeing row when the PR has no entry at all, where "not needed" already reads as a sentence.
🤖 Follow-up to #5281, from the review comments there.
Comment format. A bullet per judgement replaces the table.
included,not needed, ortype: addedfor the changelog). Its explanation is kept in an HTML comment, visible only in the Markdown source.suggest type: added; PR has type: fixed), then the explanation. Confidence is shown only when it is below high.No "already triaged" check. The workflow no longer reads the PR's comments before judging, so a PR that goes draft → ready → draft → ready is judged again and gets a second comment. Authors rarely toggle draft, and a fresh look is fine. The
<!-- llm-pr-triage -->marker and theissues: readpermission are gone with it.Config. A new
choice_labelkey names the changelog type in the comment, described in the_dockey.Tested with unit tests (47 pass) and by running the script on #5306, #5305, #5303, #5296 and the merged #5026 with
qwen3.8-27b.🤖 Generated with Claude Code