Procedural Scheduling

[skovati]

• Tickets addressed: Procedural Scheduling #972
• Review: By commit
• Merge strategy: Merge (no squash)

Description

This PR expands upon #1423 and #1299 to introduce the capability for users to define scheduling procedures in Java, upload them to Aerie, include them in a scheduling specification, and execute them as part of a scheduling run.

Closes #1534

Use gov.nasa.ammos instead of gov.nasa.jpl for new packages

In the interest of reducing rework in the upcoming Great Renaming, we've chosen to move all procedural scheduling code into the ammos namespace.

Update timeline library

TODO explain this change

Use ActivityDirectiveId and ActivityInstanceId in timeline

This PR takes the improvements made by #1505 and propagates them into the timeline library.

Return initial sim results if latest is null

Previously, getting the latest simulation results would return Optional.empty() in the case where an initial simulation has been run, but no subsequent simulations have been run. This change makes the initial simulation results available through the same method as subsequent simulation results.

Goal Invocations

This PR allows the same goal_id to appear in a scheduling specification more than once.

Detailed design

Primary Key changes

scheduling_specification_goals:

primary key (goal_invocation_id),

goal_analysis:

primary key (analysis_id, goal_invocation_id)

goal_analysis_created_activities:

primary key (analysis_id, goal_invocation_id, activity_id)

goal_analysis_satisfying_activities:

primary key (analysis_id, goal_invocation_id, activity_id)

Foreign Key changes

created_activites and satisfying_activities now FK to goal_analysis instead of just scheduling_run, since goal_analysis holds the "as run" data for that goal invocation (e.g. arguments, revision).

We were duplicating this data into each of created_activites and satisfying_activites before (i.e. storing goal_revision, goal_id, etc), but I think with the addition of arguments, it makes sense to centralize this into a goal_analysis entry per goal invocation, and then have created_activites and satisying_activities be pure association tables between directives and a goal_analysis entry.

I believe this is how we had hasura metadata set up anyways, this is just codifying this into DB FKs and simplifying this scheduling_run sub-schema

Procedural Scheduling Database Changes

We introduce the notion of a goal type, which can either be EDSL or JAR. A goal definition may now be one of those types. If it is a JAR goal, it will refer to the JAR via a new uploaded_jar_id column.

We also introduce a parameter_schema column to the goal definition, and an arguments column to goal invocations in scheduling_specification_goals.

Hook to update scheduling procedure parameters

Similar to activity types, we add a hasura event trigger on insert of a new scheduling procedure. The scheduler server loads the procedure, extracts the value schema of its parameters, and updates the corresponding database entry.

Procedural Scheduling Java Implementation

Scheduling procedure annotation processor

Scheduling procedure examples

Verification

Documentation

Future work

This PR represents the procedural scheduling MVP. There is a list of features we've deemed out of scope for this work, but would like to start working on immediately after release.

[Mythicaeda]

Reminder: DB migrations are missing.

[skovati]

Yep, working on them right now using CI tests

[mattdailis]

Joel walked me through the Kotlin files. We added back isEqualTo to Duration to preserve backwards compatibility, but marked it as @Deprecated. Approval predicated on Dan successfully running another manual test.

[dandelany]

Thanks all - just to summarize the above:

• @Mythicaeda has completed review of all Non-Kotlin code & all her feedback has been resolved
• @skovati added 3 more e2e test cases to cover cases brought up in @Mythicaeda 's review
• @JoelCourtney @mattdailis @skovati & I reviewed any new Kotlin code that has been modified since the original library PR
• I ran another manual test locally using this branch + the UI branch + the banananation model + the StayWellFed goal and all seems to be working correctly

I think we're 👍 to merge this finally!