Skip to content

fix(theory): overwrite stale symlinks and fix eko loop in inherit_ekos (#264) - #265

Open
ashishsoni-ai wants to merge 3 commits into
NNPDF:mainfrom
ashishsoni-ai:fix/overwrite-stale-symlinks
Open

ashishsoni-ai wants to merge 3 commits into
NNPDF:mainfrom
ashishsoni-ai:fix/overwrite-stale-symlinks

Conversation

@ashishsoni-ai

Copy link
Copy Markdown
Contributor

Closes #264

Summary

In TheoryBuilder.inherit_ekos and TheoryBuilder.inherit_grids, new.exists() returns False when pointing to a stale/broken symlink. This causes new.unlink() to be bypassed and new.symlink_to(...) to fail with FileExistsError.

Changes

  • src/pineko/theory.py:
    • Detect and remove stale symlinks (if new.is_symlink() and not new.exists():) in inherit_grid, inherit_eko, and eko before linking.
    • In inherit_eko, change return to continue when skipping an existing EKO so subsequent EKOs for the grid are not skipped.
  • tests/test_theory.py:
    • Added unit tests for inherit_grid and inherit_eko covering new links, existing links (with and without --overwrite), stale symlinks, and multi-EKO loop continuation.

Verification

  • black --check src/pineko/theory.py tests/test_theory.py: clean
  • flake8 tests/test_theory.py: clean (0 issues)
  • pytest tests/test_theory.py: 9 passed (100%)

AI Usage

  • Harness: Antigravity IDE
  • Model: Gemini 3.8 Flash
  • Usage: Assisted in reproducing the stale symlink error conditions and drafting unit test coverage.

@felixhekhorn felixhekhorn added the bug Something isn't working label Nov 16, 2026
Comment thread src/pineko/theory.py Resolved
Comment thread src/pineko/theory.py Resolved

@felixhekhorn felixhekhorn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - let's wait for a second review

Comment thread src/pineko/theory.py Outdated
return
eko_filename.unlink()
if not self.prepare_path(eko_filename, "operator"):
return

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be a continue just like in inherit_eko.
Let's say there are two EKO's needed and one already exists, it will silently skip recomputing the second one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thank you @jekoorn! Updated line 432 to continue so subsequent EKOs in
ames are not skipped when an earlier one already exists. Added est_eko_multiple_names_continue in ests/test_theory.py to verify this.

Comment thread tests/test_theory.py
mock_new.symlink_to.assert_called_once_with(grid)


def test_inherit_grid_stale_symlink(builder, tmp_path):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we ok in general with doing mock tests?

Also, it assers now unlink.assert_called_once(). But the bug wasn't "unlink wasn't called", it was symlink_to() raising an error. the mock just passes over the actual symptom and checks a proxy for it.
I think we should od it the same way in other tests, where a fake grid is called rather than a unittest.mock

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completely agree @jekoorn. Checking unlink.assert_called_once() was indeed only testing a proxy rather than the actual failure mode (symlink_to() raising FileExistsError).

In 2a7a288:

  1. Replaced unittest.mock.MagicMock with FakeSymlinkPath: its symlink_to() directly reproduces OS behavior by raising FileExistsError if called while the destination still exists or is a stale symlink (unless unlink() was called first).
  2. Added end-to-end filesystem tests on tmp_path (test_inherit_grid_real_filesystem and test_inherit_eko_real_filesystem) that create actual stale symlinks on disk and verify that prepare_path() cleanly clears them so symlink_to() succeeds without error (guarded by a runtime symlink capability probe so they execute fully in Linux CI).
  3. Added test_eko_multiple_names_continue to test loop continuation in TheoryBuilder.eko().

@ashishsoni-ai

Copy link
Copy Markdown
Contributor Author

Thanks @felixhekhorn! Sounds good, standing by for the second review.

@felixhekhorn

Copy link
Copy Markdown
Contributor

note that a second review has appeared

…vior

- In TheoryBuilder.eko, change return to continue when prepare_path returns False, ensuring subsequent EKOs for the grid are not skipped if an earlier EKO already exists.
- In tests/test_theory.py, replace MagicMock with FakeSymlinkPath simulating actual FileExistsError on unhandled symlink_to, and add real filesystem tests on tmp_path.
- Add test_eko_multiple_names_continue verifying the continue behavior in eko().
@ashishsoni-ai

Copy link
Copy Markdown
Contributor Author

Updated in 2a7a288:

  • Loop continuation in TheoryBuilder.eko(): Changed early return to continue when prepare_path() indicates an operator already exists, ensuring any subsequent operators in names are processed as expected.
  • Testing: Replaced MagicMock with FakeSymlinkPath which actively triggers FileExistsError on symlink_to() if the destination path was not unlinked, added real filesystem end-to-end tests on tmp_path, and added unit test coverage for multiple EKO loop continuation.
  • Verification: Verified with pytest tests/test_theory.py (14/14 passing), full test suite (34 passed), black, and flake8.

Ready for review! @felixhekhorn @jekoorn

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Overwrite if encounters stale symlinks to ekos

3 participants