AIP-84 Migrate /object/grid_data from views to FastAPI

[bugraoz93]

Description

Migrate views /object/grid_data to Fast API.

Use case/motivation

AIP-84

Related issues

Relates to: #42366

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[bbovenzi]

I mentioned in the main epic how we want to refactor this endpoint. I think we can have graph_data handle all of the task order and structure as I mentioned in #42367. Then we can simplify grid_data to only include dag run history. Each dag run includes an array of all its task instances.

Also, it would be nice to calculate summaries of mapped tasks and task groups (determine the overall state, give a breakdown of counts of children states, determine earliest queued, earliest start_date and latest end_date).

By separting runs from the task structure we can add pagination and filtering to thedag_runs_with_task_summaries endpoint, then we can do stuff like only refresh on active dag runs.

[bugraoz93]

@bbovenzi I am not sure if I grasp all the parts. I will work on this and create a PR soon. Let's discuss in the PR to make this work how you imagined

[bbovenzi]

That's alright. This is a complicated endpoint. Something like:

[{
  version_number?: number
  data_enterval_end?: string | null;
  data_interval_start?: string | null;
  end_date?: string | null;
  run_id: string;
  run_type: string;
  queued_at?: string | null;
  start_date?: string | null;
  state: string;
  task_instances?: TaskInstanceSummary[];
}]

Where the TaskInstanceSummary can be either a single task instance, or summarizing multiple tasks as a Task Group or Mapped Task. For a summary, we need:

  end_date: string | null; // latest end date
  overall_state: string | null // Check `priority` in `airflow/www/utils.py`
  start_date: string | null; // earliest start date
  queued_at: string | null // earliest queued at date
  task_count: number; // total number of children tasks inside this group or mapped task
  states: // dict of states and the number of tasks with each state
  task_id: string;
  try_number: number;

[bugraoz93]

Thanks for the details! It's clear now.