cases: Experiments

[dberenbaum]

Child issue of #2266

Proposed requirements

• Identify potential opportunities to use experiments
• Show value added by experiments

Questions

• Is this needed as a separate section?
• Are example projects needed for this?
• Who is the target user to read this over other exp docs?

[jorgeorpinel]

Ideas for slightly more specific use cases:

• Hyperspace exploration [Tuning/Optimization] (examples + params & metrics) — too narrow
• Experiment Bookkeeping? (with Git) Rapid iterations
• Visualizing Data Science (experiments + metrics/plots)

@dmpetrov and @shcheklein probably have better suggestions.

[jorgeorpinel]

This one was probably useful too! Esp. #2270 (comment) (for me at least)

[dberenbaum]

I was thinking we could open new issues when we have specific use cases in mind ready to develop, but no problem if having this open as a placeholder with some ideas is helpful for you.

[jorgeorpinel]

From #2490 (comment):

Idea: Experiments tracking/management - here we should sell W&B, MlFlow, etc - rapid iterations, live metrics + other metrics + navigation

I'm still unclear whether an experiment-related use case would be a higher priority than a ML model-related one (#2490), But the ML model direction has been waiting for a use case for a longer time.

Cc @shcheklein

[jorgeorpinel]

Closing in favor of #2544 epic.