[Pillar II ESM] Review the node allocation with PyCOMPSs and Hecuba

[kinow]

We noticed that the simulation worked with 1 member and 144 nodes, and with 2 members and 288 nodes. These number of nodes match the grid values we have from the reference mesh from FESOM2.

This workflow contained 2 tasks, one to init the simulation, that finishes quickly after creating directories¹. The other task runs fesom.x with MPI. The fesom.x Slurm job was given all the number of nodes.

With pruning, there were two PyCOMPSs tasks running in parallel. The simulation task, running fesom.x using all the nodes, and a second small task connecting to Cassandra and querying for the experiment ID every 15 seconds, checking if it needs to prune the ensemble member.

The issue is that when the pruning task started, the fesom.x @mpi + @task PyCOMPs task was never executed. The solution was to give PyCOMPSs always an extra node, and remove it from the fesom.x task. While this solved the pruning, it could be simpler, perhaps.

Footnotes

1. Maybe that task could be replaced by a PyCOMPSs @epilog, but I am not sure if the would be able to pass the expid. ↩