-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
error adding new index raster in MODIStsp #163
Comments
Thanks for reporting. Will have a look ASAP. In the meantime, could you please tell me which version of MODIStsp you are using , and share your sessioninfo() ? Lorenzo |
Hello Lorenzo. Thanks for answering. Below is the summary of the processing.
|
Hi @maximomenezes , would you be able to save your processing options to json (using "save options" from the gui, and send me the file? (You can just save it, open it in a text editor and cut and paste the text here) Lorenzo |
Hello Lorenzo. I'll check it again, but it looks like I've found a solution using the scale factor (* 0.0001). I put the formula as follows (b2_NIR *0.0001)-(b1_Red *0.0001). I tried the following and it did not work either: Once again thank you for the help and if you find a direct solution, be sure to let me know. graciously |
Hello Lorenzo. Appearing the "Apply Scale / Offset" option also generates visible images using the following formula: (b2_NIR) - (b1_Red) However, the maximum and minimum values seem confusing, in addition to doubling the image size (90mb). It seems the values are being nullified during the calculation and the use of these factors seems to avoid this. If I find any other information, I will share it. graciously |
That's weird, but I may have a hunch. two things:
"C:\OSGeo4W64\bin\gdal_translate.exe" --version and post the results? PS: Regarding file size, you can limit it using the "compression" option |
Are those result computed with "apply scale offset" to "yes" or to "no" ? |
this is what I currently get on my machine: with apply scale and offset on No and on Yes, respectively, so it appears to be identcal to arcgis (thankfully...). I suspect that this could be a GDAL version issue: are you able to run the "C:\OSGeo4W64\bin\gdal_translate.exe" --version command ? (you can also use the terminal from within RStudio, if you are using it) |
Hello Lorenzo. Going back on what I said in the last comment:
I did not inspect all the pixels, but in theory, some of them should have different values (depending on the maximum and minimum values). graciously |
Concerning the min/max: I suspect those are not coming from the "full" image. In QGIS, if I do not modify it manually, they are computed from a subset to save time. again in a terminal, can you try:
(after changing the path in the second part to that where you saved the MODIStsp results) and check the last lines? You should get something like:
(or the same, multiplied by 10000) Could you do the same check on MODIStsp outputs created with the "normal" formula (b1_Red-b2_NIR), and with Apply scale offest on "No" and then on "Yes"? (Please make sure that you use new output folders each time, otherwise, if you do not set "Reprocess existing data" to "Yes", the ouputs will not be recomputed. |
Hello Lorenzo. The questions and answers were a little overlapping. While I was doing some tests you answered and asked a few more things. When I went to answer there were other questions. Anyway, I did as you asked:
Note that by checking "yes" the image is generated normally, however, the maximum and minimum values are the same as the image generated in ArcGis without having multiplied by the scale factor. |
Hi Maximo. Yes, we were working concurrently so there was a bit of a mess. Sorry for that I think may have pinpointed the problem. It woul be however useful if you may be able to run the gdalinfo command u shared in my last on the file resulting from using apply scale offset = no. |
It seems to me that the script is applying a division for the value 10000 even though we did not select any option. When we mark "yes" in the scale factor, division and multiplication are canceled by generating a file with the value of simple subtraction. |
Hi. I agree. And i think that is related to a breaking change in latest gdal versions, since on tests on my machine yesterday all seemed to be ok. I will be able to check maybe tomorrow. My interest in the min/max values for the no scale offset is in understanding which value you get in the image, besides the Minmax values (according to your last, you get 0 and 1, whch makes sense because in that case we save as integers). Thanks for all your input! I'll keep you posted |
Hello Lorenzo. A partial solution is to check "yes" in the "AppyScale / Offset" option. This circumvents the script by nullifying the undue division. And next to that, use the formula with the correction factor when adding the new index ((b2_NIR) - (b1_Red)) * 0.0001. With this I got the same values as the ArcGis Raster Calculator (see figure). |
Thanks for the workaround. However, this is not expected bahaviour, so it will be fixed ASAP (I am updating my gdal now to 2.4, to see if i can replicate the problem). |
Ok. I confirmed this is a GDAL version issue. For future reference, this was most probably due to this change: https://trac.osgeo.org/gdal/ticket/3221#comment:5 Working on a fix. |
Hello Lorenzo. I am more of a user than a programmer. I do not know if I understood correctly, but the script or the gdal would be applying scale factor with wrong values or changed in the MODIS product bands. Sorry if I'm being superficial, but internalizing a table (like this one below) would not prevent the program from making the wrong choice? For example, when we choose an option in the "Category" menu it directs and limits the "product" option to the one we chose in the previous option. Can not do something similar with different scale factor? When choosing an option the script would already filter and direct to the respective scale factor. Sorry if I misunderstood. My English is still basic. Origin of information: http://modis-sr.ltdri.org/guide/MOD09_UserGuide_v1_3.pdf |
This is what MODIStsp already does, and on systems with GDAL < 2.3.x it works well. However, due to a recent change in the However, I now have a tentative fix for this. Could you please:
require(remotes)
remotes::install_github("ropensci/MODIStsp", ref = "feature/fix_gdalvrtbug") Let me know if it works. |
Hello Lorenzo. Failed to install one of the packages / dependencies, but it appears that the installation has completed. I tested simple subtraction and generated a normal image! Thank you so much for the help! graciously Máximo |
Thanks to you for reporting! I will push the update also to CRAN as soon as possible. |
Lorenzo ...
If you can permanently add the MIRBI ((10 * b7_SWIR) - (9.8 * b6_SWIR) +2)
index to M * 09GA product I will be grateful.
I'm waiting for more updates.
graciously
Máximo
Bibliography:
https://pdfs.semanticscholar.org/82f6/6727eab9fc26d272b2fb129058a331669a45.pdf
|
Hi Lorenzo, I was having similar issue with the scaling as described by Máximo. I am following your comment in which you asked to remove modistsp and reinstall : require(remotes) After running the lines above, the R window gives the following options. Which one do I need to choose? " 1: R.utils (2.7.0 -> 2.8.0 ) [CRAN] 2: R6 (2.3.0 -> 2.4.0 ) [CRAN] 3: raster (2.6-7 -> 2.8-19) [CRAN] " Thanks, |
Hi @osamasajid123 , Unless you need to keep some older versions of a package, I'd suggest to select "All" (10). That way, you'll bring all packages up to current versions. Lorenzo |
Hi Lorenzo, I did what you said. Now when I try to open the interface by running MODIStsp(), I get the following error. Can you understand what is going wrong? Warning in read.dcf(file.path(p, "DESCRIPTION"), c("Package", "Version")) : Do you authorize this? 1: Yes - "MODIStsp_Previous.json" will be saved permanently and updated after 2: No - Previous options will be written to Choice (1/2): 1 |
Strange. I'd try restarting r and trying install again on a fresh session. |
Hello.
My name is Máximo Menezes. I am a user of the MODIStsp script and am having trouble adding new indexes.
For example, I intend to perform simple subtraction of band 2 (NIR) with band 1 (Red). This operation fails even though I try different ways of writing the formula.
I tried without relatives: b2_NIR-b1_Red, with parentheses (b2_NIR-b1_Red), with space and unrelated b2_NIR - b1_Red and with parentheses and space (b2_NIR - b1_Red).
None of these mathematical operations accuse error (Set New Index), however, none of them generate a visible image. The file is generated, however, in white.
Could you help me understand what would be wrong?
Thanks and I await the response.
Best regards
Máximo Menezes Costa
The text was updated successfully, but these errors were encountered: