Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execute new commondata generation in CI #2099

Merged
merged 26 commits into from
Nov 2, 2024
Merged

Execute new commondata generation in CI #2099

merged 26 commits into from
Nov 2, 2024

Conversation

Radonirinaunimi
Copy link
Member

@Radonirinaunimi Radonirinaunimi commented Jun 4, 2024

Addresses #1457 but also:

  • makes sure that all the filter.py works properly
  • makes sure that a default formatter is used when dumping floating numbers

Remaining to do:

  • fail if the exit status of even one single filter is non-zero (which currently happens a lot)
  • fail if there are modified and/or unstaged files in nnpdf_data/nnpdf_data/new_commondata
  • list all the changed commondata files in comments

@Radonirinaunimi Radonirinaunimi marked this pull request as draft June 4, 2024 22:21
@Radonirinaunimi Radonirinaunimi force-pushed the check_newcd branch 2 times, most recently from 44f6938 to 47c71b5 Compare June 4, 2024 23:22
@Radonirinaunimi Radonirinaunimi marked this pull request as ready for review June 5, 2024 10:49
add manual trigger

add workflow

revert mistakenly deleted file
add filter dependencies as exras & remove underscore
@RoyStegeman RoyStegeman mentioned this pull request Jun 27, 2024
34 tasks
@Radonirinaunimi
Copy link
Member Author

There are a few datasets that have changed. Many of them are due to numerical fluctuations, so #2095 needs to be propagated.

List of datasets that have been modified
nnpdf_data/nnpdf_data/new_commondata/ATLAS_2JET_7TEV_R06/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/ATLAS_2JET_7TEV_R06/uncertainties_stronger.yaml
nnpdf_data/nnpdf_data/new_commondata/ATLAS_2JET_7TEV_R06/uncertainties_weaker.yaml
nnpdf_data/nnpdf_data/new_commondata/ATLAS_Z0_8TEV_ZMASS/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/CMS_1JET_8TEV/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/CMS_2JET_7TEV/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/CMS_WPWM_13TEV_ETA/uncertainties_WM.yaml
nnpdf_data/nnpdf_data/new_commondata/CMS_WPWM_13TEV_ETA/uncertainties_WP.yaml
nnpdf_data/nnpdf_data/new_commondata/CMS_WPWM_7TEV_ELECTRON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/CMS_WPWM_7TEV_MUON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/CMS_WPWM_8TEV_MUON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/CMS_Z0_7TEV_DIMUON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/H1_1JET_319GEV_290PB-1_DIF/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/H1_1JET_319GEV_290PB-1_DIF/uncertainties_norm.yaml
nnpdf_data/nnpdf_data/new_commondata/H1_2JET_319GEV_290PB-1_DIF/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/H1_2JET_319GEV_290PB-1_DIF/uncertainties_norm.yaml
nnpdf_data/nnpdf_data/new_commondata/HERMES_NC_7GEV_ED/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/HERMES_NC_7GEV_EP/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_DY_7TEV_MUON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_DY_8TEV_MUON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_WPWM_7TEV_MUON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_WPWM_8TEV_MUON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_Z0J_13TEV_2022/uncertainties_dimuon_pT.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_Z0_13TEV/uncertainties_dielectron.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_Z0_13TEV/uncertainties_dimuon.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_Z0_13TEV_2022/uncertainties_dimuon_y.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_Z0_7TEV_DIELECTRON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_Z0_7TEV_MUON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_Z0_8TEV_DIELECTRON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/LHCB_Z0_8TEV_MUON/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2009_1JET_200GEV/uncertainties_CC.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2009_1JET_200GEV/uncertainties_CF.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2009_2JET_200GEV/uncertainties_A.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2009_2JET_200GEV/uncertainties_B.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2009_2JET_200GEV/uncertainties_C.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2009_2JET_MIDRAP_200GEV/uncertainties_OS.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2009_2JET_MIDRAP_200GEV/uncertainties_SS.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2012_1JET_510GEV/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2012_2JET_510GEV/uncertainties_A.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2012_2JET_510GEV/uncertainties_B.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2012_2JET_510GEV/uncertainties_C.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2012_2JET_510GEV/uncertainties_D.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2013_1JET_510GEV/uncertainties.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2013_2JET_510GEV/uncertainties_A.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2013_2JET_510GEV/uncertainties_B.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2013_2JET_510GEV/uncertainties_C.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2013_2JET_510GEV/uncertainties_D.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2015_1JET_200GEV/uncertainties_CC.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2015_1JET_200GEV/uncertainties_CF.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2015_2JET_MIDRAP_200GEV/uncertainties_OS.yaml
nnpdf_data/nnpdf_data/new_commondata/STAR_2015_2JET_MIDRAP_200GEV/uncertainties_SS.yaml

Copy link
Member Author

@Radonirinaunimi Radonirinaunimi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the datasets one-by-one, the changes in the uncertainties are due to:

There are only a few datasets (CMS_1JET_8TEV and some of the STAR data [cc @giacomomagni]) for which there are changes not attributed to one of the above.

@scarlehoff
Copy link
Member

@Radonirinaunimi do we want to try to merge this before going through the actual data?

@Radonirinaunimi
Copy link
Member Author

@Radonirinaunimi do we want to try to merge this before going through the actual data?

Sorry for the late reply, for some reasons I have not received notifications about this.

Yes, let me resolve these merge conflicts tomorrow and then we can try to merge. As far as I recall, there is only one single dataset that I need to look really closer.

@scarlehoff scarlehoff added the run-fit-bot Starts fit bot from a PR. label Nov 1, 2024
Copy link
Member

@scarlehoff scarlehoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for this. This is really fantastic.

I've added the fitbot flag thinking about maybe small changes but I don't actually think it is necessary seeing that the regression tests have not changed.

We might want to wait until it finish since it is already running though.

Copy link

github-actions bot commented Nov 1, 2024

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

@scarlehoff scarlehoff added run-fit-bot Starts fit bot from a PR. and removed run-fit-bot Starts fit bot from a PR. labels Nov 1, 2024
Copy link

github-actions bot commented Nov 1, 2024

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

Copy link
Member

@scarlehoff scarlehoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@scarlehoff scarlehoff added run-fit-bot Starts fit bot from a PR. and removed run-fit-bot Starts fit bot from a PR. labels Nov 1, 2024
@scarlehoff scarlehoff linked an issue Nov 1, 2024 that may be closed by this pull request
Copy link

github-actions bot commented Nov 1, 2024

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

@scarlehoff scarlehoff merged commit 425ef48 into master Nov 2, 2024
7 checks passed
@scarlehoff scarlehoff deleted the check_newcd branch November 2, 2024 08:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Don't save more decimals than necessary in the uncertainty (or data) files Execute data generation in the CI
3 participants