Tools for CF (Roadmap paper)

[lhmarsden]

Topic for discussion

This discussion is related to the theme of what tools CF should promote/contribute to/develop in the discussed roadmap paper for the next 5 years.

Tools that have been discussed are

• CF checker https://github.com/cedadev/cf-checker (requires some development)
• Schema
• Libraries

[lhmarsden]

https://github.com/cf-convention/cf-convention.github.io/blob/main/software.md page lists various tools

[gtesoro]

Schema discussion https://github.com/orgs/cf-convention/discussions/357

[lhmarsden]

I would like to see a CF checker developed and endorsed. There are a few out there already. If we endorsed one perhaps the community would know which to contribute to.

[gtesoro]

Hopefully netCDF-Schema can fill that gap. I've updated the development thread on this repo. Aiming for a 1.0.0 before the end of the year so we could asses if it enough to provide a replacement for the CF Checker.

[lhmarsden]

How generic will this netCDF-Schema be? Do you aim to cover all of the CF conventions and different profiles of data that one might envisage within this? This would be a very ambitious project! Or something more specific to the profiles of data that you are working with?

I think this is somewhere that the CF checker (understandably) falls short. There are many degrees of freedom within CF, and there needs to be in order to cover alot of different types of data. But it is consequently difficult to develop a generic checker that is useful for each type of data one might have.

[gtesoro]

I've updated the info about the concept here, please take a look.

Since the idea is to have a generic validation model that can accept different schemas (including CF and other subsets) users could customize which parts of the schema they want to keep and which ones to expand uppon.

[lhmarsden]

Perhaps relevant to this, I developed with @filchos a spreadsheet template generator with a configuration for CF-NetCDF. It is designed to help people who prefer to work usually with spreadsheets get their data into some more structured format before they start converting them. The user can select column headers from the full list of CF standard names, and there is a separate sheet for global attributes. I would like to add a separate sheet for variable attributes too at some point.

The template generator is hosted here:
https://www.nordatanet.no/aen/template-generator/config%3DCF-NetCDF

The code is here (anyone reading is welcome to provide feedback using issues or we can work together on developing it further)
https://github.com/SIOS-Svalbard/Nansen_Legacy_template_generator

We published a paper on this over the summer:
https://doi.org/10.5334/dsj-2024-038

It might be that we work on a converter to CF-NetCDF at some point, probably starting with some low hanging fruit e.g. time series or vertical profiles.

[lhmarsden]

We have discussed some of tools we currently have available and that are in development but not how we should discuss tools in the roadmap paper.

Should this be as simple as stating that tools will be listed on the website?
https://github.com/cf-convention/cf-convention.github.io/blob/main/software.md

What approach should we have towards adding tools to the website? Should they be reviewed by someone in CF in the same way proposals for standard names are reviewed? Do we have criteria for which tools we will list? Do we want a wish list of tools from the community, and use the roadmap to ask for more community driven efforts to help us get there?