@skovati, @Mythicaeda, @AaronPlave & I met a few times re: some outstanding questions about the UX for procedural constraints, and the impacts of those decisions on UI & DB implementation.

The main issue is related to questions around cacheability, ordering, and idempotence of procedural constraints. We are walking a fine line here, because on one hand, it's possible that the constraints provided by users will not be idempotent and could have side effects that are order-dependent or timing-dependent (see NASA-AMMOS/aerie#1609 for a discussion of this problem with goals) - and we want to provide them with the "escape hatch" needed to deal with these scenarios. On the other hand, we very much want to encourage users to write idempotent, order-free constraints, and we want to keep the same visual language around "checked" vs "unchecked" constraints - ie. a constraint that has already been run on a given plan with given inputs should be considered "checked" even though technically it could be non-idempotent and return a different result if re-run.

With these considerations in mind, we're proposing the following user experience:

• Constraints should have an order to them
  • this is a useful feature for organization as much as anything
  • we will execute them in this order when user runs constraints
  • BUT UI for ordering constraints should not be identical to Goals,
    • we shouldn't show it as "priority", it implies more than we want.
    • instead, should be a simple drag and drop to reorder, like the one used for ordering layers in the Timeline Editor
• Constraint runs will still be cached and mostly follow the same logic around "checked" vs. "unchecked", with some small differences:
  • To be considered "checked" (ie. "not stale") in the UI, the client must check that the constraint:
    • is in the constraint run table (ie. has a cached result)
    • and has the same arguments as the previous run (since constraints have args now)
  • As a result, if constraints have been re-ordered but nothing else has changed (same plan, same arguments), they should still be considered "checked" and not stale.
• There should be a "force re-run" button in the UI to force constraints to re-run even if they're all "checked"
  • Similar to what we do for re-running simulation
  • This is the main escape hatch for users running non-idempotent constraints

I think this is an effective compromise between the two worlds, but feel free to discuss if there are other ideas for how to handle this more simply. The main things that need to be implemented for this on the UI side are:

• drag and drop re-ordering for constraints in the spec (& sending new ordering to hasura)
• updated staleness check logic (to check arguments + cache table)
• "re-run constraints" button in the UI which calls the normal constraint run with a new "force" flag (TBD, will be added on DB PR)

Plan is to slice ^these changes in half and @duranb focus on just the force re-run button for now, potentially delay ordering for a future issue.

The second bullet is also required for release - specifically we should make sure arguments are being considered for the staleness check - if arguments have changed, should be "unchecked"