Skip to content

Cover overshooting final hops in the judge - #805

Open
jengyoyoAo wants to merge 2 commits into
sysprog21:mainfrom
jengyoyoAo:codex/fix-drone-pad-hopping
Open

jengyoyoAo wants to merge 2 commits into
sysprog21:mainfrom
jengyoyoAo:codex/fix-drone-pad-hopping

Conversation

@jengyoyoAo

@jengyoyoAo jengyoyoAo commented Nov 21, 2026 •

Copy link
Copy Markdown

Problem and behavior

The existing Drone Pad Hopping cases let a solution that always subtracts one final hop pass. For example, it returns 0 for [2,1], where the answer is 1.

Approach

Add three cases where the reachable range extends past the final pad, along with a regression test that checks each case rejects the reported solution. Regenerate the judge file and update the contract pins and golden digest.

Validation

node --test tests/browser/compiler-explorer.test.js: 108 passed, 0 failed, 0 skipped.

python3 scripts/gen-problems.py --check: passed.

./scripts/test.sh: completed in an ARM64 Linux VM with exit code 0 and no failed gates. Used ShellCheck 0.9.0 to match the Ubuntu 24.04 CI package.

Chromium/ffmpeg-dependent tests, cargo-audit, cargo-deny, and actionlint were skipped because the required tools were unavailable. The x86_64-only Clang-fetch test was also skipped.

Close #797


Summary by cubic

Closes a gap in the jump-game-ii judge that let incorrect solutions pass by always subtracting a final hop, such as returning 0 for [2, 1] when the answer is 1.

  • Adds three cases where the reachable range overshoots the final pad, each pinning the correct expected value.
  • Adds a browser regression test that checks each case accepts the correct greedy solution and rejects the reported final-hop subtraction, plus a contract entry for each.
  • Regenerates the judge file and updates the contract pins and golden digest.

Written for commit d7ea5de. Summary will update on new commits.

View guided diff

Existing cases let an incorrect final-hop subtraction pass.
Add cases that reject it while preserving correct solutions.

Close sysprog21#797

Change-Id: I7804cb50bc1eceab8678fb93d2d608eae9ddc682
cubic-dev-ai[bot]

This comment was marked as resolved.

}
});

test("jump-game-ii cases reject subtracting a final hop unconditionally", () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This repeats the shape of jump-game cases reject each hop-by-hop shortcut (look each label up, assert.ok(pinned), compare the wrong answer with expected), and the open judge-gap PRs repeat either it (#779) or the anagram test's any-case form (#777, #761). As a follow-up beyond this PR, one helper and one table would make the next judge fix a data-only change: a row per shortcut, checked against every pinned case, or against any case when the row names no label. Run over the anagram, jump-game and this test's shortcuts at this head it passes all 8, and it fails exactly where these tests do when the three overshooting cases are removed, when one of them is changed to an input the shortcut gets right, or when the anagram's expected: false cases are dropped.

import { isDeepStrictEqual } from "node:util";

const assertCasesReject = (id, name, wrong, labels) => {
  const { cases } = judges[id];
  const caught = ({ input, expected }) =>
    !isDeepStrictEqual(wrong(...input), expected);
  if (!labels) {
    assert.ok(cases.some(caught), `the built-in ${id} cases must reject ${name}`);
    return;
  }
  for (const label of labels) {
    const pinned = cases.find((row) => row.label === label);
    assert.ok(pinned, `the built-in cases must keep "${label}"`);
    assert.ok(caught(pinned), `"${label}" must reject ${name}`);
  }
};

for (const [id, name, wrong, labels] of SHORTCUTS) {
  test(`${id} cases reject ${name}`, () => assertCasesReject(id, name, wrong, labels));
}

A new shortcut still needs its wrong solution written, but then the row is all it needs.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! A shared helper and table would make these tests easier to maintain. I'll leave that for a follow-up and keep this PR focused on #797.

"isomorphic-strings": "65993b8137658baf9c57dd17847f3edc30d51481e3194e05940ce91e2e8ff26b",
"jump-game": "a01ddbb3aea588dc8fa943d5c4655459d7502b7477eae92d6cb025aba249cb80",
"jump-game-ii": "e08cc621d3a69f16d95b48bd1f8c6212b33638cc8d48a69c79abac1aa5d3c44a",
"jump-game-ii": "ffa7555442cd9130d1694fd58031b279308860cdea7b82e3f8b648641c62d2c0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every judge fix edits this digest by hand, and unlike tests/golden/task-prompts.json (UPDATE_PROMPT_GOLDEN=1) and the problem golden (UPDATE_PROBLEM_GOLDEN=1), docs/adding-a-problem.md gives no way to refresh it or says it needs one. As a follow-up beyond this PR, an UPDATE_POSED_GOLDEN=1 switch on test_posed_bank_golden_matches_python_judges_starters_and_notes in tests/test_task_package.py could write it with the posed and compact the test already uses, which reproduce this value (ffa75554…) from the bank at this head. every_public_judge_and_starter_matches_the_instructor_preview in tests/unit/tasks/exercise.rs would still check the Rust side against the written file, and the change would still show in the diff for review.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion. I'll leave this as a follow-up and keep this PR focused on #797.

ColtenOuO

This comment was marked as resolved.

Reject incorrect expected values as well as the reported shortcut.
Check each pinned case with a correct greedy solution.

Change-Id: Ic0cf849942c7d6ab30d2a1b57c4d88957f844544
@jserv

jserv commented Nov 21, 2026

Copy link
Copy Markdown
Contributor

Some review comments on this pull request are not settled yet. Please refine the change per the review comments listed below, reply on each thread with what you changed or why you did not, and mark each thread resolved yourself once it is settled (Resolve conversation, at the bottom of the thread). The pull request stays blocked while review threads are open.

Not addressed at the current head:

Checked at 13ea6cb.

@chenyu50 chenyu50 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed d7ea5de on macOS arm64 with Node v26.13.0.

Focused validation:

$ node --test --test-name-pattern='jump-game-ii' tests/browser/compiler-explorer.test.js
1 passed, 0 failed, 0 skipped.

$ python3 scripts/gen-problems.py --check
Passed.

The reported final-hop subtraction is already rejected at this head. The two inline comments suggest additional test coverage:

  • A solution that rejects zero-valued destinations when nums.length > 1 still passes all eight built-in cases. Changing the two-pad case to [2,0], with expected result 1, rejects that shortcut while preserving the final-hop subtraction check.
  • With the existing correct-solution assertion retained, replacing the third case with [4,1,1,1] and expected: 1 leaves the current regression passing. The proposed first-hop-only assertion passes with the original case and fails after that replacement, preserving coverage of overshooting after the first hop.

Comment thread problem-bank/judges.json
"input": [
[
2,
1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a small extension of this case, all eight cases still accept an implementation that returns -1 when nums.length > 1 && nums.at(-1) === 0, then otherwise uses the correct greedy algorithm.

Replacing this 1 with 0 gives [2,0] -> 1. It still rejects the reported final-hop subtraction and also checks that reaching a zero-valued destination is valid.

Suggested change
1
0

for (const label of [
"two pads with room to spare",
"three pads with room to spare",
"range grows after the first pad",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the existing correct-solution assertion in place, keeping this label but replacing its case with [4,1,1,1] and expected: 1 still leaves the regression green and removes coverage of overshooting after the first hop.

The variant (nums) => nums.length === 1 ? 0 : nums[0] >= nums.length - 1 ? 1 : reported(nums) passes the other seven cases but fails the current [1,3,1,1] case.

Check that named variant against this pinned case as well, so the later-hop coverage cannot be replaced by another first-hop case.

One concrete approach is to add this after byLabel is built, before the existing loop:

const firstHopOnly = (nums) =>
  nums.length === 1
    ? 0
    : nums[0] >= nums.length - 1
      ? 1
      : reported(nums);

const later = byLabel["range grows after the first pad"];
assert.ok(later);
assert.notEqual(
  firstHopOnly(later.input[0]),
  later.expected,
  "must reject a first-hop-only repair",
);

This passes with the current [1,3,1,1] case. Replacing that case with [4,1,1,1] and expected: 1 makes this assertion fail, preserving the later-hop coverage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Drone Pad Hopping accepts a solution that undercounts an overshooting last hop

4 participants