--restore-session --failed doesn't seem to work with --repeat

[mirenradia]

When I run a benchmark with a command such as

reframe -c /path/to/my/benchmark -r --repeat=2

and one of the runs fails, doing

reframe --restore-session --failed -r

finds 0 checks so I see

[==========] Running 0 check(s)
[==========] Started on Wed Dec  6 11:37:46 2023

[----------] start processing checks
[----------] all spawned checks have finished

[  PASSED  ] Ran 0/0 test case(s) from 0 check(s) (0 failure(s), 0 skipped, 0 aborted)
[==========] Finished on Wed Dec  6 11:37:46 2023

This doesn't seem to be a problem when one of the runs fails without the --repeat flag.

Please find attached two consecutive ReFrame reports demonstrating this issue:

• run-report-186.json (a session which has a failed run)
• run-report-187.json (an attempt to restore the session and rerun the failed run)

[vkarak]

This is a limitation actually of all the options that dynamically generate new tests, such as --repeat, --distribute and --parameterize. There are two problems that prevent restoring such tests:

1. They don't have a corresponding file from which they could be reloaded and restored.
2. Even if you pass the -c along with the --restore-session in order to force reframe to look into a specific path for tests, no test will be restored either. The reason is that these automatically generated tests are technically different from their parents (they are derived parameterized tests which simply have the same base name). So reframe cannot match them to any of the loaded tests in order to restore them.

We will need to update the docs to reflect this behaviour.

Now, if you really want session restore to support automatically generated tests, I suggest opening a separate feature request.

[mirenradia]

Now, if you really want session restore to support automatically generated tests, I suggest opening a separate feature request.

OK, I've created a separate issue for the feature request: #3077.