Add External Events

[pranav-super]

___REQUIRES_AERIE_PR___="1513"

A new construct in AERIE is that of External Events. In this PR, we will introduce External Events, their containing entities (External Sources), their grouping mechanisms (Derivation Groups), as well as how they manifest in the UI and in the backend. Extensive documentation on this will be added to aerie-docs, but a general introduction can be provided in this PR description.

External Events

An External Event is a construct that represents the occurrence of something outside of what is modeled in the mission model. They could be events that are close to the spacecraft and directly affect behavior on it, like a DSN visibility event that might affect when communications happen, or they could be much more remote, like a constellation event. Functionally, these are at the interstice between external datasets and activities, in that they manifest on the timeline as activities (discrete blocks of time), but they themselves do not affect the mission model or invoke any actions much like external datasets.

The need for these became apparent after an exploration of External Events implemented as activities and as resources. To mimic the same construct as a resource, we would need to parse the external data, and then place it on the timeline as a discrete resource, which given the fact that there are gaps between events and there can also be simultaneous events, proved extremely unwieldy and awkward. While activities were a more natural representation, we still required an intermediary of a resource to schedule these activities based off of an ingested file, as it isn't possible to schedule activities exclusively following information in a file.

As such, a new construct became envisioned, and that is what is introduced in this PR.

How?

External Events are bundled in files called External Sources. External Sources additionally contain some metadata about the collection of External Events (i.e. what period of time this External Source applies over). These can be uploaded to AERIE and managed in a newly added External Source Manager:

It is in this view that External Sources can be inspected, uploaded, and deleted. Both the metadata as well as the component External Events (via a Timeline or Table view) can be inspected:

External Events as a Table	External Events as a Timeline

Furthermore, grouped within another new construct (and operation) known as Derivation Groups, these External Sources and their External Events can be viewed on a timeline:

For the sake of brevity, we will not discuss these details further. Furthermore, details on the derivation operation can be found in the AERIE PR, as it is there that the majority of derivation-related logic is handled.

Discussion Items and TODOS:

Discussion Items

• Squash tables for source type, event type, and derivation group?
• Discuss valid_at - should it be called valid_after, created_at, published_at, etc.
• Parsing can/should be merged to Aerie Gateway to use ‘common’ parsing code?
• Ideal branching behavior?
• Cherrypicking of events/individual replacement of events outside of window?

TODOS:

• Bugs:
  • UpdateCard in plan persists very unnecessary extra data somehow.
  • Tooltip becomes permanently sticky for some reason on the timeline (seen on ExternalSourceManager.svelte)
  • Verify that during file parsing/upload, it is just one transaction? It seems fine to create an external event type and then fail on creation of a source, so the ONLY thing we should worry about here is verifying that orphaned events (events without a source) are impossible
  • Sometimes nothing appears in the external sources panel - rare timing issue.
• Features/Other:
  • Remove cleanup - we can let source types and event types linger. Simply verify that empty derivation groups/types can still be readded to and that we don't create a new one. Should have a way to delete these if they are empty?
  • make valid_at adhere to 8061. actually all dates if possible
  • Tests for 0 duration events
  • Just parse and then discard the files!!!
  • (MAYBE) change derivation group in the UI itself? (not yet?)
  • (MAYBE) Add handling in ExternalSourceManager if can't reach hasura, don't show toasts saying it worked and switch panels, instead fail out, give a message, cancel the outgoing request?
  • (MAYBE) add handling about views refreshing for user change in plan
  • (MAYBE) make sure filters select correctly in ExternalSourceManger.svelte on user change, add tests
  • Change spacing in timeline for ee's if not grouping, to not justify to bottom but just to add height+5 space between rows
  • fix name of query GetExternalEvents in gql.ts, update docs
  • make view for external_source_event_type just a graphql query instead as this is faster

[duranb]

Do you mind reverting all the changes made to the port and github workflows, please?

[pranav-super]

Do you mind reverting all the changes made to the port and github workflows, please?

To be specific, do you mean all changes in .github/workflows? Or are there additional places too?
Also, I believe that in test.yml we needed the parts about caching; is that okay to leave in?

[duranb]

Do you mind reverting all the changes made to the port and github workflows, please?

To be specific, do you mean all changes in .github/workflows? Or are there additional places too? Also, I believe that in test.yml we needed the parts about caching; is that okay to leave in?

I'd say just the added caching test step in the test.yml file is fine to keep, but all the other changes in that folder in general do not seem necessary to support the feature. I'm open to discussing more about it if you feel otherwise. The only changes outside that folder that should be reverted are the port changes from 3000 -> 3005.

[pranav-super]

I'd say just the added caching test step in the test.yml file is fine to keep, but all the other changes in that folder in general do not seem necessary to support the feature. I'm open to discussing more about it if you feel otherwise. The only changes outside that folder that should be reverted are the port changes from 3000 -> 3005.

Taken care of!

[duranb]

Everything looks good to me! Thank you!