Missing GRIB-1 time range indicators #1246
Labels
bug
Something isn't working
Comments
|
Hi Robert:
The way that works is that a Grib1Customizer customizes behavior when its
not standard WMO GRib1. So if you look at NcepTables you see it does handle
timeRangeIndicator=130. OTOH, if its copernicus, its probably not NCEP so
you'll have to track down the correct tables, since theres no reason to
assume that they are using the same definition as NCEP. Which is why we
cant just "look around" for some table with timeRangeIndicator=130.
In any case, you have to look at the grib record, figure out what table it
needs, then configure the code to recognize what table to use for that file:
https://docs.unidata.ucar.edu/netcdf-java/current/userguide/grib_tables.html#grib-1-table-handling
If you want to post the file, Im sure the Unidata folks can help. If
its huge, and you know how (ToolsUI can do it, but its obscure), you can
extract a sample record that shows the problem.
Regards,
John
…On Sun, Oct 8, 2023 at 8:15 PM R. Schmunk ***@***.***> wrote:
Versions impacted by the bug
I am working with a devo copy of netCDF-Java 5.3.4, snapshot downloaded
early August.
What went wrong?
A user pinged me because netCDF-Java (and thus Panoply) would not open a
GRIB-1 dataset, with an exception thrown because the data use time range
indicator 130. See stack trace below.
In looking at the netCDF-Java code, I see that Grib1ParamTime rejects and
time range indicator with value > 125. However, in looking for more info
about this, I found that there are at least thirteen time range indicators
greater than that limit, as shown on e.g. this NOAA/NCEP webpage
<https://www.nco.ncep.noaa.gov/pmb/docs/on388/table5.html>
Although the above link is to a NOAA webpage, the user originally
downloaded the data from an EU Copernicus site
<https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels-monthly-means?tab=overview>.
Unfortunately, I don't know what specific field input he used there to
acquire the problem dataset. Instead he provided me a copy of the file.
Is there any chance that netCDF-Java developers can look into this in the
foreseeable future? I haven't looked at the code in Grib1ParamTime
closely enough to see if I could puzzle it out for myself, and bugs in
another app are calling for my immediate attention.
Relevant stack trace
Caused by: java.lang.IllegalArgumentException: PDS: Unknown Time Range Indicator 130
at ucar.nc2.grib.grib1.Grib1ParamTime.<init>(Grib1ParamTime.java:399)
at ucar.nc2.grib.grib1.tables.Grib1Customizer.getParamTime(Grib1Customizer.java:118)
at ucar.nc2.grib.grib1.Grib1Record.getParamTime(Grib1Record.java:124)
at ucar.nc2.grib.collection.Grib1CollectionBuilder.filterIntervals(Grib1CollectionBuilder.java:160)
at ucar.nc2.grib.collection.Grib1CollectionBuilder.makeGroups(Grib1CollectionBuilder.java:107)
at ucar.nc2.grib.collection.GribCollectionBuilder.createMultipleRuntimeCollections(GribCollectionBuilder.java:128)
at ucar.nc2.grib.collection.GribCollectionBuilder.createIndex(GribCollectionBuilder.java:120)
at ucar.nc2.grib.collection.GribCdmIndex.openGribCollectionFromDataFile(GribCdmIndex.java:821)
at ucar.nc2.grib.collection.GribCdmIndex.openGribCollectionFromDataFile(GribCdmIndex.java:804)
at ucar.nc2.grib.collection.GribCdmIndex.openGribCollectionFromRaf(GribCdmIndex.java:774)
at ucar.nc2.grib.collection.GribIosp.build(GribIosp.java:183)
at ucar.nc2.NetcdfFiles.build(NetcdfFiles.java:815)
at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:754)
at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:276)
Relevant log messages
*No response*
If you have an example file that you can share, please attach it to this
issue.
If so, may we include it in our test datasets to help ensure the bug does
not return once fixed?
Note: the test datasets are publicly accessible without restriction.
Yes
Code of Conduct
- I agree to follow the UCAR/Unidata Code of Conduct
—
Reply to this email directly, view it on GitHub
<#1246>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEVZQCZBVNMJ5Z6I3PIYDTX6NM5XAVCNFSM6AAAAAA5YD2PESVHI2DSMVQWIX3LMV43ASLTON2WKOZRHEZTEMJZGQZDSNI>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
@JohnLCaron, Depends on your definition of huge. The sample file I was provided is 170 MB. |
|
hmm i dont see the file |
|
@JohnLCaron, I didn't include a link in my original post about this. Here's a Dropbox link. |
|
And today I get a complaint from someone downloading data from the same Copernicus website that they are getting an "Unknown Time Range Indicator 128" exception trying to open the download. Sorry, no sample file provided this time. |
|
Its clear that ECMWF has started using some non-standard grib1 time
parameters.
Someone has to go find out what they are, and perhaps also check on whether
the parameter tables have changed.
Its fair to ask your customer to do so, presumably they are the ones who
care, and perhaps might know enough to tell if they are right.
Then send that information into Unidata support, who should be able to add
the code to implement it.
This will happen for as long as you try to support GRIB (esp GRIB1), so you
could try to standardize it, and point people at a FAQ or something. Or you
could stop trying to support GRIB1.
We did make a deliberate decision to let things break, rather than allow
incorrect data. But we could add a debug flag allowing people to see the
data, and set the time range to something like Jan 1, 1970.
It also occurs to me that you are looking at a legacy product for
which there is a newer, maybe GRIB2 equivalent.
…On Wed, Oct 11, 2023 at 9:30 AM R. Schmunk ***@***.***> wrote:
And today I get a complaint from someone downloading data from the same
Copernicus website that they are getting an "Unknown Time Range Indicator
128" exception trying to open the download. Sorry, no sample file provided
this time.
—
Reply to this email directly, view it on GitHub
<#1246 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEVZQH7SO4RWOAXXETYPTDX623PPANCNFSM6AAAAAA5YD2PEQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Versions impacted by the bug
I am working with a devo copy of netCDF-Java 5.5.4, snapshot downloaded early August.
What went wrong?
A user pinged me because netCDF-Java (and thus Panoply) would not open a GRIB-1 dataset, with an exception thrown because the data use time range indicator 130. See stack trace below.
In looking at the netCDF-Java code, I see that
Grib1ParamTimerejects and time range indicator with value > 125. However, in looking for more info about this, I found that there are at least thirteen time range indicators greater than that limit, as shown on e.g. this NOAA/NCEP webpageAlthough the above link is to a NOAA webpage, the user originally downloaded the data from an EU Copernicus site. Unfortunately, I don't know what specific field input he used there to acquire the problem dataset. Instead he provided me a copy of the file.
Is there any chance that netCDF-Java developers can look into this in the foreseeable future? I haven't looked at the code in
Grib1ParamTimeclosely enough to see if I could puzzle it out for myself, and bugs in another app are calling for my immediate attention.Relevant stack trace
Relevant log messages
No response
If you have an example file that you can share, please attach it to this issue.
If so, may we include it in our test datasets to help ensure the bug does not return once fixed?
Note: the test datasets are publicly accessible without restriction.
Yes
Code of Conduct
The text was updated successfully, but these errors were encountered: