Skip to content

Commit

Permalink
[uss_qualifier/scenarios/netrid] Factor away test step fragment for f…
Browse files Browse the repository at this point in the history
…light presence from scenario (#862)
  • Loading branch information
mickmis authored Dec 12, 2024
1 parent 6784504 commit 424a6b4
Show file tree
Hide file tree
Showing 9 changed files with 68 additions and 56 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -489,6 +489,8 @@ def _evaluate_flight_presence(
mapping_by_injection_id: Dict[str, TelemetryMapping],
verified_sps: Set[str],
):
"""Implements fragment documented in `display_data_evaluator_flight_presence.md`."""

query_timestamps = [q.request.timestamp for q in observation_queries]
observer_participants = (
[observer_participant_id] if observer_participant_is_relevant else []
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# ASTM NetRID v19 flight presence test step fragment

This fragment is implemented in `display_data_evaluator.py:RIDObservationEvaluator._evaluate_flight_presence`.

#### Premature flight check

The timestamps of the injected telemetry usually start in the future. If a flight with injected telemetry only in the future is observed prior to the timestamp of the first telemetry point, this check will fail because the SP does not satisfy **[interuss.automated_testing.rid.injection.ExpectedBehavior](../../../../requirements/interuss/automated_testing/rid/injection.md)**.

#### Missing flight check

**[astm.f3411.v19.NET0610](../../../../requirements/astm/f3411/v19.md)** requires that SPs make all UAS operations discoverable over the duration of the flight plus *NetMaxNearRealTimeDataPeriod*, so each injected flight should be observable during this time. If a flight is not observed during its appropriate time period, this check will fail.

**[astm.f3411.v19.NET0710,1](../../../../requirements/astm/f3411/v19.md)** and **[astm.f3411.v19.NET0340](../../../../requirements/astm/f3411/v19.md)** require a Service Provider to implement the GET flights endpoint. This check will also fail if uss_qualifier cannot query that endpoint (specified in the ISA present in the DSS) successfully.

The identity of flights is determined by precisely matching the known injected positions. If the flight can be found, the USS may not have met **[astm.f3411.v19.NET0260,Table1,9](../../../../requirements/astm/f3411/v19.md)** or **[astm.f3411.v19.NET0260,Table1,10](../../../../requirements/astm/f3411/v19.md)** prescribing provision of position data consistent with the common data dictionary.

#### Lingering flight check

**[astm.f3411.v19.NET0260,NearRealTime](../../../../requirements/astm/f3411/v19.md)** requires a SP to provide flights up to *NetMaxNearRealTimeDataPeriod* in the past, but an SP should preserve privacy and ensure relevancy by not sharing flights that are further in the past than this window.
Original file line number Diff line number Diff line change
Expand Up @@ -62,21 +62,7 @@ If a DSS was provided to this test scenario, uss_qualifier acts as a Display Pro

**[astm.f3411.v19.NET0250](../../../../requirements/astm/f3411/v19.md)** requires that a NetRID Service Provider rejects a request for a very large view area with a diagonal greater than *NetMaxDisplayAreaDiagonal*. If such a large view is requested and a 413 error code is not received, then this check will fail.

#### Premature flight check

The timestamps of the injected telemetry usually start in the future. If a flight with injected telemetry only in the future is observed prior to the timestamp of the first telemetry point, this check will fail because the SP does not satisfy **[interuss.automated_testing.rid.injection.ExpectedBehavior](../../../../requirements/interuss/automated_testing/rid/injection.md)**.

#### Missing flight check

**[astm.f3411.v19.NET0610](../../../../requirements/astm/f3411/v19.md)** requires that SPs make all UAS operations discoverable over the duration of the flight plus *NetMaxNearRealTimeDataPeriod*, so each injected flight should be observable during this time. If a flight is not observed during its appropriate time period, this check will fail.

**[astm.f3411.v19.NET0710,1](../../../../requirements/astm/f3411/v19.md)** and **[astm.f3411.v19.NET0340](../../../../requirements/astm/f3411/v19.md)** require a Service Provider to implement the GET flights endpoint. This check will also fail if uss_qualifier cannot query that endpoint (specified in the ISA present in the DSS) successfully.

The identity of flights is determined by precisely matching the known injected positions. If the flight can be found, the USS may not have met **[astm.f3411.v19.NET0260,Table1,9](../../../../requirements/astm/f3411/v19.md)** or **[astm.f3411.v19.NET0260,Table1,10](../../../../requirements/astm/f3411/v19.md)** prescribing provision of position data consistent with the common data dictionary.

#### Lingering flight check

**[astm.f3411.v19.NET0270](../../../../requirements/astm/f3411/v19.md)** requires a SP to provide flights up to *NetMaxNearRealTimeDataPeriod* in the past, but an SP should preserve privacy and ensure relevancy by not sharing flights that are further in the past than this window.
#### [Flight presence checks](./display_data_evaluator_flight_presence.md)

#### Flights data format check

Expand Down Expand Up @@ -120,17 +106,7 @@ Per **[interuss.automated_testing.rid.observation.ObservationSuccess](../../../.

Per **[interuss.automated_testing.rid.observation.UniqueFlights](../../../../requirements/interuss/automated_testing/rid/observation.md)**, the same flight ID may not be reported by a Display Provider for two flights in the same observation.

#### Premature flight check

The timestamps of the injected telemetry usually start in the future. If a flight with injected telemetry only in the future is observed prior to the timestamp of the first telemetry point, this check will fail because the SP does not satisfy **[interuss.automated_testing.rid.injection.ExpectedBehavior](../../../../requirements/interuss/automated_testing/rid/injection.md)**.

#### Missing flight check

**[astm.f3411.v19.NET0610](../../../../requirements/astm/f3411/v19.md)** require that SPs make all UAS operations discoverable over the duration of the flight plus *NetMaxNearRealTimeDataPeriod*, so each injected flight should be observable during this time. If one of the flights is not observed during its appropriate time period, this check will fail.

#### Lingering flight check

**[astm.f3411.v19.NET0270](../../../../requirements/astm/f3411/v19.md)** requires a SP to provide flights up to *NetMaxNearRealTimeDataPeriod* in the past, but an SP should preserve privacy and ensure relevancy by not sharing flights that are further in the past than this window.
#### [Flight presence checks](./display_data_evaluator_flight_presence.md)

#### [Flight consistency with Common Data Dictionary checks](./common_dictionary_evaluator_dp_flight.md)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# ASTM NetRID v22a flight presence test step fragment

This fragment is implemented in `display_data_evaluator.py:RIDObservationEvaluator._evaluate_flight_presence`.

## Premature flight check

The timestamps of the injected telemetry usually start in the future. If a flight with injected telemetry only in the future is observed prior to the timestamp of the first telemetry point, this check will fail because the SP does not satisfy **[interuss.automated_testing.rid.injection.ExpectedBehavior](../../../../requirements/interuss/automated_testing/rid/injection.md)**.

## Missing flight check

**[astm.f3411.v22a.NET0610](../../../../requirements/astm/f3411/v22a.md)** requires that SPs make all UAS operations discoverable over the duration of the flight plus *NetMaxNearRealTimeDataPeriod*, so each injected flight should be observable during this time. If a flight is not observed during its appropriate time period, this check will fail.

**[astm.f3411.v22a.NET0710,1](../../../../requirements/astm/f3411/v22a.md)** and **[astm.f3411.v22a.NET0340](../../../../requirements/astm/f3411/v22a.md) require a Service Provider to implement the GET flights endpoint. This check will also fail if uss_qualifier cannot query that endpoint (specified in the ISA present in the DSS) successfully.

The identity of flights is determined by precisely matching the known injected positions. If the flight can be found, the USS may not have met **[astm.f3411.v22a.NET0260,Table1,10](../../../../requirements/astm/f3411/v22a.md)** or **[astm.f3411.v22a.NET0260,Table1,11](../../../../requirements/astm/f3411/v22a.md)** prescribing provision of position data consistent with the common data dictionary.

## Lingering flight check

**[astm.f3411.v22a.NET0260,NearRealTime](../../../../requirements/astm/f3411/v22a.md)** requires a SP to provide flights up to *NetMaxNearRealTimeDataPeriod* in the past, but an SP should preserve privacy and ensure relevancy by not sharing flights that are further in the past than this window.
Original file line number Diff line number Diff line change
Expand Up @@ -62,21 +62,7 @@ If a DSS was provided to this test scenario, uss_qualifier acts as a Display Pro

**[astm.f3411.v22a.NET0250](../../../../requirements/astm/f3411/v22a.md)** requires that a NetRID Service Provider rejects a request for a very large view area with a diagonal greater than *NetMaxDisplayAreaDiagonal*. If such a large view is requested and a 413 error code is not received, then this check will fail.

#### Premature flight check

The timestamps of the injected telemetry usually start in the future. If a flight with injected telemetry only in the future is observed prior to the timestamp of the first telemetry point, this check will fail because the SP does not satisfy **[interuss.automated_testing.rid.injection.ExpectedBehavior](../../../../requirements/interuss/automated_testing/rid/injection.md)**.

#### Missing flight check

**[astm.f3411.v22a.NET0610](../../../../requirements/astm/f3411/v22a.md)** requires that SPs make all UAS operations discoverable over the duration of the flight plus *NetMaxNearRealTimeDataPeriod*, so each injected flight should be observable during this time. If a flight is not observed during its appropriate time period, this check will fail.

**[astm.f3411.v22a.NET0710,1](../../../../requirements/astm/f3411/v22a.md)** and **[astm.f3411.v22a.NET0340](../../../../requirements/astm/f3411/v22a.md) require a Service Provider to implement the GET flights endpoint. This check will also fail if uss_qualifier cannot query that endpoint (specified in the ISA present in the DSS) successfully.

The identity of flights is determined by precisely matching the known injected positions. If the flight can be found, the USS may not have met **[astm.f3411.v22a.NET0260,Table1,10](../../../../requirements/astm/f3411/v22a.md)** or **[astm.f3411.v22a.NET0260,Table1,11](../../../../requirements/astm/f3411/v22a.md)** prescribing provision of position data consistent with the common data dictionary.

#### Lingering flight check

**[astm.f3411.v22a.NET0270](../../../../requirements/astm/f3411/v22a.md)** requires a SP to provide flights up to *NetMaxNearRealTimeDataPeriod* in the past, but an SP should preserve privacy and ensure relevancy by not sharing flights that are further in the past than this window.
#### [Flight presence checks](./display_data_evaluator_flight_presence.md)

#### Flights data format check

Expand Down Expand Up @@ -122,17 +108,7 @@ Per **[interuss.automated_testing.rid.observation.ObservationSuccess](../../../.

Per **[interuss.automated_testing.rid.observation.UniqueFlights](../../../../requirements/interuss/automated_testing/rid/observation.md)**, the same flight ID may not be reported by a Display Provider for two flights in the same observation.

#### Premature flight check

The timestamps of the injected telemetry usually start in the future. If a flight with injected telemetry only in the future is observed prior to the timestamp of the first telemetry point, this check will fail because the SP does not satisfy **[interuss.automated_testing.rid.injection.ExpectedBehavior](../../../../requirements/interuss/automated_testing/rid/injection.md)**.

#### Missing flight check

**[astm.f3411.v22a.NET0610](../../../../requirements/astm/f3411/v22a.md)** require that SPs make all UAS operations discoverable over the duration of the flight plus *NetMaxNearRealTimeDataPeriod*, so each injected flight should be observable during this time. If one of the flights is not observed during its appropriate time period, this check will fail.

#### Lingering flight check

NET0260 requires a SP to provide flights up to *NetMaxNearRealTimeDataPeriod* in the past, but an SP should preserve privacy and ensure relevancy by not sharing flights that are further in the past than this window.
#### [Flight presence checks](./display_data_evaluator_flight_presence.md)

#### [Flight consistency with Common Data Dictionary checks](./common_dictionary_evaluator_dp_flight.md)

Expand Down
7 changes: 6 additions & 1 deletion monitoring/uss_qualifier/suites/astm/netrid/f3411_19.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<th><a href="../../README.md#checked-in">Checked in</a></th>
</tr>
<tr>
<td rowspan="74" style="vertical-align:top;"><a href="../../../requirements/astm/f3411/v19.md">astm<br>.f3411<br>.v19</a></td>
<td rowspan="75" style="vertical-align:top;"><a href="../../../requirements/astm/f3411/v19.md">astm<br>.f3411<br>.v19</a></td>
<td><a href="../../../requirements/astm/f3411/v19.md">DSS0010</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/netrid/v19/dss/token_validation.md">ASTM NetRID DSS: Token Validation</a></td>
Expand Down Expand Up @@ -251,6 +251,11 @@
<td>Implemented</td>
<td><a href="../../../scenarios/astm/netrid/v19/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../../requirements/astm/f3411/v19.md">NET0260,NearRealTime</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/netrid/v19/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../../requirements/astm/f3411/v19.md">NET0260,NetSpDataResponseTime95thPercentile</a></td>
<td>Implemented</td>
Expand Down
7 changes: 6 additions & 1 deletion monitoring/uss_qualifier/suites/astm/netrid/f3411_22a.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<th><a href="../../README.md#checked-in">Checked in</a></th>
</tr>
<tr>
<td rowspan="99" style="vertical-align:top;"><a href="../../../requirements/astm/f3411/v22a.md">astm<br>.f3411<br>.v22a</a></td>
<td rowspan="100" style="vertical-align:top;"><a href="../../../requirements/astm/f3411/v22a.md">astm<br>.f3411<br>.v22a</a></td>
<td><a href="../../../requirements/astm/f3411/v22a.md">DSS0010</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/netrid/v22a/dss/token_validation.md">ASTM NetRID DSS: Token Validation</a></td>
Expand Down Expand Up @@ -256,6 +256,11 @@
<td>Implemented</td>
<td><a href="../../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../../requirements/astm/f3411/v22a.md">NET0260,NearRealTime</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../../requirements/astm/f3411/v22a.md">NET0260,NetSpDataResponseTime95thPercentile</a></td>
<td>Implemented</td>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
<th><a href="../README.md#checked-in">Checked in</a></th>
</tr>
<tr>
<td rowspan="99" style="vertical-align:top;"><a href="../../requirements/astm/f3411/v22a.md">astm<br>.f3411<br>.v22a</a></td>
<td rowspan="100" style="vertical-align:top;"><a href="../../requirements/astm/f3411/v22a.md">astm<br>.f3411<br>.v22a</a></td>
<td><a href="../../requirements/astm/f3411/v22a.md">DSS0010</a></td>
<td>Implemented</td>
<td><a href="../../scenarios/astm/netrid/v22a/dss/token_validation.md">ASTM NetRID DSS: Token Validation</a></td>
Expand Down Expand Up @@ -252,6 +252,11 @@
<td>Implemented</td>
<td><a href="../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../requirements/astm/f3411/v22a.md">NET0260,NearRealTime</a></td>
<td>Implemented</td>
<td><a href="../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../requirements/astm/f3411/v22a.md">NET0260,NetSpDataResponseTime95thPercentile</a></td>
<td>Implemented</td>
Expand Down
7 changes: 6 additions & 1 deletion monitoring/uss_qualifier/suites/uspace/required_services.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<th><a href="../README.md#checked-in">Checked in</a></th>
</tr>
<tr>
<td rowspan="99" style="vertical-align:top;"><a href="../../requirements/astm/f3411/v22a.md">astm<br>.f3411<br>.v22a</a></td>
<td rowspan="100" style="vertical-align:top;"><a href="../../requirements/astm/f3411/v22a.md">astm<br>.f3411<br>.v22a</a></td>
<td><a href="../../requirements/astm/f3411/v22a.md">DSS0010</a></td>
<td>Implemented</td>
<td><a href="../../scenarios/astm/netrid/v22a/dss/token_validation.md">ASTM NetRID DSS: Token Validation</a></td>
Expand Down Expand Up @@ -253,6 +253,11 @@
<td>Implemented</td>
<td><a href="../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../requirements/astm/f3411/v22a.md">NET0260,NearRealTime</a></td>
<td>Implemented</td>
<td><a href="../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td><a href="../../requirements/astm/f3411/v22a.md">NET0260,NetSpDataResponseTime95thPercentile</a></td>
<td>Implemented</td>
Expand Down

0 comments on commit 424a6b4

Please sign in to comment.