Update check for load user modulefiles to allow exact match

[jo-basevi]

Allow checks for "modules: load:" modulefiles to allow when there's an exact match in available modules. This is to fix bug #481 where module avail for openmpi/4.1.5 includes openmpi/4.1.5 and openmpi/4.1.5-debug.

I've also added tests for parsing module avail output.

Closes #481.

@aidanheerdegen last week there was an issue with multiple modules found in /g/data/vk83/modules and /g/data/vk83/modules/access-models, even though they were the same modulefile. I noticed in the modules docs there's a module paths command that lists paths of available modulefiles matching a pattern so for the above, it would still evaluate to the same path. There's a couple of differences to module avail:

• Running the command module paths using modulecmd ($MODULESHOME//bin/modulecmd) generates a bunch of code. So for bash, it generates a bunch of echo statements, rather than output going to stderr which is the case for module avail. So to get what values are printed, have to run the generated code, and then capture the output.
• Specifying openmpi/ with a / just matches the default module version (I think), so only one path is returned.

If it's unlikely to have a case where module/version can be accessed from two different modulepaths and are different, e.g.

/path/to/modulefiles:
test-module/1.0.0
/another/module/path:
test-module/1.0.0

The check could be changed from modules_avail.count(modulefile) != 1 to modulefile not in modules_avail in this PR. Maybe this be moved to separate to avoid blocking this PR?

[pep8speaks]

Hello @jo-basevi! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2024-08-13 23:31:54 UTC

[coveralls]

coverage: 52.612% (+0.3%) from 52.281%
when pulling bfecc12 on ACCESS-NRI:481-fix-payu-module-uniqueness-check
into 83001a2 on payu-org:master.

[aidanheerdegen]

Some comments ... but happy to merge if they're ignorable.

[aidanheerdegen]

last week there was an issue with multiple modules found in /g/data/vk83/modules and /g/data/vk83/modules/access-models, even though they were the same modulefile

That was an implementation bug. We've removed the access-models subdirectory to fix it. So it isn't a use case we need to care about or support. Does that help?

[jo-basevi]

Thanks for the review!

That was an implementation bug. We've removed the access-models subdirectory to fix it. So it isn't a use case we need to care about or support. Does that help?

Yes, that does help thanks. Ok so don't have to worry about using module paths or changing the implementation in that case.

[aidanheerdegen]

LGTM.