Skip to content

[release/2.1] Proper fixes for EPIC batch install script - #2211

Open
EdwardSnyder-NOAA wants to merge 3 commits into
JCSDA:release/2.1from
EdwardSnyder-NOAA:batch-install-fixes
Open

[release/2.1] Proper fixes for EPIC batch install script#2211
EdwardSnyder-NOAA wants to merge 3 commits into
JCSDA:release/2.1from
EdwardSnyder-NOAA:batch-install-fixes

Conversation

@EdwardSnyder-NOAA

@EdwardSnyder-NOAA EdwardSnyder-NOAA commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Use of generative AI

Please refer to our temporary spack-stack AI policy for further information.

  • I confirm that this contribution was submitted by a real person. If AI tools or agents were used in its preparation, I have personally reviewed and take responsibility for all submitted content, code, and proposed changes. This contribution is not autonomously generated by AI.

Description

A few git commits from the dev branch to get the batch_install.sh script working with EPIC GitHub Actions workflow for the release/2.1 branch.

Dependencies

None.

Issues addressed

None.

Applications affected

None.

Systems affected

None.

Testing

  • CI: Note whether the automatic tests (GitHub actions tests that run automatically for every commit) pass or not
    • GitHub actions CI tests pass
    • GitHub actions CI tests do not pass (provide explanation)
    • GitHub actions CI tests skipped (provide explanation if necessary)
  • New tests added: List and describe any new tests added to GitHub actions
    • ...
  • Additional testing: Add information on any additional tests conducted
    • ...

Checklist

  • This PR addresses one issue/problem/enhancement or has a very good reason for not doing so.
  • These changes have been tested on the affected systems and applications.
  • All dependency PRs/issues have been resolved and this PR can be merged.
  • All necessary updates to the documentation (spack-stack wiki) will be made when this PR is merged

rickgrubin-noaa and others added 3 commits November 10, 2026 11:16
@rickgrubin-noaa @EdwardSnyder-NOAA
Add batch install script and updated source / binary mirror files for…
3468261 
… EPIC Tier 1 hosts (JCSDA#2043)

Add a batch install script for EPIC Tier 1 hosts, and update mirrors.yaml files for those hosts to define source and binary cache locations.

File util/epic/intel_oneapi-2026.0.0_install.sh is added for potential future need / use.

Both scripts shamelessly borrowed from NRL host scripts.

(cherry picked from commit c17b15c)
@rickgrubin-noaa @EdwardSnyder-NOAA
Update EPIC batch install to fix module loading for oneAPI-based envs (…
edc6800 
…JCSDA#2115)

Add required module pathing and loading for MSU hosts to successfully build oneAPI envs. GCC envs are not effected.

Module pathing:
- specify module use pathing for the EPIC-installed oneAPI compilers
- specify module use pathing for the system-installed oneAPI MPI and MKL

Module loading:

load module git-lfs so that packages that require it are successfully fetched

This is accomplished by adding configs/sites/tier1/{hercules, orion}/setup.sh scripts and referencing them in the top-level spack setup.sh script. This allows for removing incomplete module pathing commands from util/epic/batch_install.sh and instead place them in site-specific setup scripts. Placing these updates in configs/sites/tier1/{hercules, orion}/setup.sh ensures that they're available to both manual and automated builds.

Note that a module pathing command for site derecho in util/epic/batch_install.sh was removed because it's already in configs/sites/tier1/derecho/setup.sh

(cherry picked from commit b2f2346)
@EdwardSnyder-NOAA
Add compiler build argument to EPIC batch install script (JCSDA#2183)
5bfd3e1 
(cherry picked from commit 0f6a74d)
@climbfuji climbfuji changed the title Proper fixes for EPIC batch install script [release/2.1] Proper fixes for EPIC batch install script Nov 10, 2026
@climbfuji climbfuji self-assigned this Nov 10, 2026
RatkoVasic-NOAA
RatkoVasic-NOAA approved these changes Nov 11, 2026
View reviewed changes

@RatkoVasic-NOAA RatkoVasic-NOAA left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@natalie-perlin

natalie-perlin commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

@EdwardSnyder-NOAA - thank you for making this change! How did you test these approaches?

@EdwardSnyder-NOAA

EdwardSnyder-NOAA commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

@EdwardSnyder-NOAA - thank you for making this change! How did you test these approaches?

For the past few weeks, I have been using the batch_install.sh script from the develop branch to build spack-stack-2.1.1 in my spack-stack GitHub Actions workflow. If you do a history of this file in the develop branch, you will see I'm bringing in those commits to the file here in the release/2.1 branch. I excluded your commit and Matt's CARGO_HOME rename commit since that was pertaining to the release/2.2 branch. Basically I'm trying to sync up the batch_install.sh file from the release/2.1 branch with the file in the develop branch.

@natalie-perlin

natalie-perlin commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

@EdwardSnyder-NOAA - thank you for making this change! How did you test these approaches?

For the past few weeks, I have been using the batch_install.sh script from the develop branch to build spack-stack-2.1.1 in my spack-stack GitHub Actions workflow. If you do a history of this file in the develop branch, you will see I'm bringing in those commits to the file here in the release/2.1 branch. I excluded your commit and Matt's CARGO_HOME rename commit since that was pertaining to the release/2.2 branch. Basically I'm trying to sync up the batch_install.sh file from the release/2.1 branch with the file in the develop branch.

@EdwardSnyder-NOAA - do you mean the commit from #2210?
It would apply to all branches, not just release/2.2 (it is not there yet; still pending the update). Bootstrapping mirrors would likely fail IF 1) you already have prior mirrors, and you create a new spack-stack version as a separate directory (e.g., spack-stack-2.2.0), and 2) the bootstrap tag has changed in spack. I am not sure what the outcome would be from combining these factors... My attempt to bootstrap a mirror on Ursa for a new spack-stack version has resulted in an error, as the prior installation already populated the mirror metadata.

@EdwardSnyder-NOAA

EdwardSnyder-NOAA commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

@EdwardSnyder-NOAA - thank you for making this change! How did you test these approaches?

For the past few weeks, I have been using the batch_install.sh script from the develop branch to build spack-stack-2.1.1 in my spack-stack GitHub Actions workflow. If you do a history of this file in the develop branch, you will see I'm bringing in those commits to the file here in the release/2.1 branch. I excluded your commit and Matt's CARGO_HOME rename commit since that was pertaining to the release/2.2 branch. Basically I'm trying to sync up the batch_install.sh file from the release/2.1 branch with the file in the develop branch.

@EdwardSnyder-NOAA - do you mean the commit from #2210? It would apply to all branches, not just release/2.2 (it is not there yet; still pending the update). Bootstrapping mirrors would likely fail IF 1) you already have prior mirrors, and you create a new spack-stack version as a separate directory (e.g., spack-stack-2.2.0), and 2) the bootstrap tag has changed in spack. I am not sure what the outcome would be from combining these factors... My attempt to bootstrap a mirror on Ursa for a new spack-stack version has resulted in an error, as the prior installation already populated the mirror metadata.

@natalie-perlin - I was referring to commit #2210. Are you planning on cherry picking this commit to release/2.1 then? I'm fine having this commit to future release branches. I'd figure since we are versioning control the batch_install.sh script it is okay to leave it from the release/2.1 branch since that script was working before your commit. That is why I excluded it from this PR. If I add your commit, then I'll have to go back and rerun everything, which I guess isn't a big deal since everything is automated at this point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@RatkoVasic-NOAA RatkoVasic-NOAA RatkoVasic-NOAA approved these changes
@natalie-perlin natalie-perlin Awaiting requested review from natalie-perlin

At least 2 approving reviews are required to merge this pull request.

Assignees

@climbfuji climbfuji

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@EdwardSnyder-NOAA @natalie-perlin @RatkoVasic-NOAA @climbfuji @rickgrubin-noaa