plots: allow definition of plots section as list

[dtrifiro]

Top-level section for plots currently requires a mapping, which looks like this:

# dvc.yaml
plots:
  evaluation/test/plots/confusion_matrix.json: # Configure template and axes.
    template: confusion
    x: actual
    y: predicted
  logs.csv:
    x: actual
    y: [predicted_class, predicted_class2]

with this PR, we allow for the plots top-level section to be defined as a list, which should make it more similar to how plots are defined in the stage sections:

# dvc.yaml
plots:
  - evaluation/test/plots/confusion_matrix.json: # Configure template and axes.
      template: confusion
      x: actual
      y: predicted
  - logs.csv:
      x: actual
      y: [predicted_class, predicted_class2]

[dtrifiro]

Somehow related to #8371, since in that case I'd expect params/metrics to be lists and not mappings

[dberenbaum]

@pared What do you think? I think this was purposely a mapping since unique keys are required, but isn't that true for stage plots and other deps/outs that take lists?

[pared]

@dberenbaum yep, I think that we can enforce uniqueness with schema validation rules. Also that would make plots more conforming with how we handle outputs.

[dberenbaum]

@dtrifiro Great idea. Let's do it!

[dberenbaum]

Somehow related to #8371, since in that case I'd expect params/metrics to be lists and not mappings

Agreed, I don't think we should need mappings support for these.

[daavoo]

Reminder to update https://github.com/iterative/dvcyaml-schema

[dtrifiro]

Thanks @daavoo: iterative/dvcyaml-schema#23

[dtrifiro]

@dberenbaum can we merge this?

[dberenbaum]

Let's merge, although I don't have permission to merge

[skshetry]

What is the canonical way of writing top-level plots now?

Although I prefer this schema, I am -1 on this as I don't see much difference between the two, and only increases the maintenance burden and worsens the error message (voluptuous falls back to the first schema on Any, giving confusing error message). We are offering two similar formats to define plots.

Also, the top-level plots section is not the same as stages.<stage>.plots section and felt like a conscious choice to me. The plots section is also very overloaded:

plots: Union[
      Dict[
          PlotIdOrFilePath, Union[TopLevelPlotFlags, EmptyTopLevelPlotFlags]
      ],
      List[
          Dict[
              PlotIdOrFilePath,
              Union[TopLevelPlotFlags, EmptyTopLevelPlotFlags],
        ]
    ],
]

Regarding 3.0 deprecation, we haven't deprecated anything in dvc.yaml, and we are most likely stuck with that schema.

[dberenbaum]

What is the canonical way of writing top-level plots now?

I think list format should be the canonical way. It matches stage plots and stage outputs and also upcoming top-level sections like params and metrics.

We need to update docs here. We can update now to only mention list support. We can even only add list support to schema validation.

Not sure how much it saves us to make the breaking change in 3.0 of dropping mapping/dict support. I'm fine with leaving it unless it will cause major headaches so as not to break anyone's projects.