Adding Model Data Init and Training Progress to MosaicMLLogger

[jjanezhang]

Adding Model Data Init and Training Progress to MosaicMLLogger

1. Logs timestamp when model data is downloaded and initialized
2. Logs training progress metrics whenever we log to metadata

Testing
Started a run and confirmed that training metrics and data initialized are in run's metadata

[dakinggg]

curious what @mvpatel2000 and @eracah think, but do we need a new callback for this? That is, do we actually want to log this to all logger destinations? or should we just have the MosaicMLLogger do this?

[mvpatel2000]

I think it's fine to log to one or all logger destinations -- the others just won't really care since and the UX is clean on most of them. I've done one offs like this using the logger API and not had issues.

I'm not sure this belongs in Composer though. The callbacks in Composer should be foundational callbacks which are broadly applicable. This feels like a callback for a bespoke use case which probably better belongs in whatever repo is leveraging it? @jjanezhang feel free to start a conversation in the composer channel if you feel it's appropriate

[irenedea]

Nice! Had some comments and questions

[irenedea]

Looks good, awesome work! Just some nits and wanted to double check that moving to batch_end is the right call 😄

[mvpatel2000]

Requesting sign off from me for progress logic before this lands. Can review fast whenever Jane requests :)

[mvpatel2000]

@jjanezhang will review when tests pass :)

[mvpatel2000]

LGTM! Sorry this took a bit to review