Scheduler and Task Creation improvements

[seriousben]

Context

Fixes:

• If a reducer parent and a reducer task finish at the same time, the scheduler will not know about it which might result in more than one reduced output.
• If a reducer failed for an output, we should stop scheduling new reducers while we wait for all parent tasks to complete. (This is will show up as error processing state change: No outputs found for previous reducer task, should never happen)

What

• We skip creating or enqueuing a new reducer task, if the compute node has failed tasks.
  • New unit test covers this case.
• We only do process and persist one state change at a time to prevent timing issues.
  • Existing unit test was changed to test this.

Testing

Contribution Checklist

• If the python-sdk was changed, please run make fmt in python-sdk/.
• If the server was changed, please run make fmt in server/.
• Make sure all PR Checks are passing.

[seriousben]

This is fixing a regression introduced in #1072