Payu current run info/log

[jo-basevi]

It can be hard to tell where payu is up to during a payu run, especially when the model is being run sequentially -n N, and there's multiple different PBS jobs being run (the main model execution job, and then subsequent post-processing jobs - collate, sync, postscripts).

A command (payu status/payu info/?) could display:

• what jobs are currently being run or have completed.
• map to the standard error/output logs files for each job.
• If jobs exits with an error - it can be hard to tell if a post-processing job exits with an error unless manually reading all the log files.

The above information could be stored in log file that is populated as various stages are submitted and running.

If jobs are currently being run, payu sweep and payu checkout could be prevented from running. Related Issues:

• Hold sweep while job is on the queue: sweeping while a job is on the queue #44
• Running sweep before a checkout - has some ideas on naming log files with UUIDs: Sweep before changing branches #418