Azure metadata tags (resource labels) propagation for child jobs

[abhi18av]

New feature

With the recent improvements to resource labelling d64eeff and #2853 , it'd be great if the child jobs can be tagged on Azure Batch executor as well.

Usage scenario

When submitting jobs to Azure Batch with Nextflow, the tags provided as the process level should be propagated to the generated job definitions.

Suggest implementation

We would need to implement the current labelling mechanism within the AzBatchService, by (probably) enhancing the createPool implementation.

The metadata tags would show up at the job metadata menu on the portal

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Takadonet]

Propagation tags to child resources will allow for cost tracking. Our need is to trace back all cost associated by group, project and individuals which tags enable.

[pditommaso]

This should be a quick win. tagging @bentsherman

[bentsherman]

Looking at AzBatchService.getOrCreatePool(), it looks like Nextflow tries to create a new pool only if an identical one doesn't already exist. So if different resource labels are specified for different processes, Nextflow should create a different pool for each one. Does that mean we need to insert the resource labels as metadata in the AzVmPoolSpec class, during specFromPoolConfig and specFromAutoPool ?

[abhi18av]

Judging from the codebase of JobOperations perhaps we should explore tagging at the job level itself and therefore, whichever pool the job gets allocated it would have that metadata 🤔

[Takadonet]

Will job level tagging appear in the Cost analysis page for the subscription? Do not believe so. If so, that would be very fine grain knowledge which be awesome.

[pditommaso]

I'm wondering if the concept of metadata in Azure is the same as tags in AWS ?

[abhi18av]

I think the Cost Analysis does allow filtering based on the tag dimension and here's the relevant information.

https://learn.microsoft.com/en-gb/azure/cost-management-billing/costs/understand-cost-mgt-data#how-tags-are-used-in-cost-and-usage-data

Perhaps @vsmalladi might be able to check/confirm this faster. But I could do some quick experimentation and circle back, I'd need to initiate a set of runs for the same.

[vsmalladi]

@abhi18av will take a look at this issue today.

[abhi18av]

Thanks mate - happy to collaborate via Nextflow slack.