[Data Issue]: backward calculation Netherlands went wrong?

[DutchPower24]

When did this happen?

Today

What zones are affected?

Nl

What is the problem?

Could it be that something went wrong? 2019 & 2020 numbers are looking very strange at the moment. Looks like fossil based energy is missing?

[Nivedhitha-dp]

I’d like to get a better understanding to help address the issue and possibly work on a solution. Can somebody clarify the following?

Are there specific data sources or calculations from 2019 and 2020 that seem to be missing or incorrect?
Could you provide details on which fossil-based energy sources should be included in the data, or if there are specific metrics that need to be recalculated?
Are there any specific tests or checks you'd like to see for verifying the data consistency going forward?

Also can you assign me to this issue, I can address it !

[FelixDQ]

Thank you for reporting @DutchPower24

@Nivedhitha-dp I think that we might just need to refetch and reprocess the data using the parser, but it would be good to validate. If you can use the parser to check if there are any problems in 2019/2020, that would be great! After you've validated the data, I can start a refetch/reprocess.

There are instructions for how to run the parser here: https://github.com/electricitymaps/electricitymaps-contrib/wiki/Fixing-a-broken-parser

Thanks!

[FelixDQ]

From a quick look, it seems like ned.nl aren't reporting gas production before 2021. Can you check if we are using their API wrong or alternatively if we can get the gas production from an alternative source for the historical data? It looks like we can get it from ENTSO-E

[DutchPower24]

I’ve checked the Ned.nl website and it looks like that solar and wind data is going back until 2017 and all the other sources only until 2021.

So it seams like that for the data before 2021 a combination has to be made between Ned.nl (solar and wind data) and entso-e for the other production types.

[annaib00]

I have updated the parser to include historic ENTSOE data before 2021 for types where there is no NED.nl data in the PR #7401. Thanks for reporting and looking into this!

[DutchPower24]

I have updated the parser to include historic ENTSOE data before 2021 for types where there is no NED.nl data in the PR #7401. Thanks for reporting and looking into this!

When will the data be updated? I see the pull request merged but the data is still wrong.

[FelixDQ]

We are currently refetching and reprocessing the data. We had to fix a few issues in the backend, but I expect the data to be available in ~1-2 weeks.

[DutchPower24]

We are currently refetching and reprocessing the data. We had to fix a few issues in the backend, but I expect the data to be available in ~1-2 weeks.

Great, looking forward.