Issue using get_nhdplushr for a few basins

[tjstagni]

I'm using the latest nhdplusTools 1.1.0 version and updated all R packages. I've downloaded nhdplusHR data using the download_nhdplushr() function

I have the following errors when trying to use get_nhdplushr() for the following basins:

"0803" "0804" "0805" "0806" "0807"
Error in rbind.data.frame(...) : numbers of columns of arguments do not match

"0108" "0110" "0107"
"0202" "0203" "0204" "0205" "0206" "0207" "0208"
"1403" "1405" "1406" "1401" "1402" "1407" "1408"
Error in select(): ! Can't select columns that don't exist. ✖ Column ReachCode doesn't exist.
This workflow has worked for several other basins

Also, I have the following error using download_nhdplushr() function for the following basins:

"0512"
Error in zip::unzip(out_file, exdir = out[length(out)]) : zip error: Cannot open zip file \AppData\Roaming\R\data\R\nhdplusTools\temp_hr_cache\05\NHDPLUS_H_0512_HU4_GDB.zip for reading in file zip.c:141

The NHDPLUS_H_0512_HU4_GDB.zip file exist in the directory above, and this function worked for other basins

reproducible workflow
`
temp_dir = file.path(nhdplusTools_data_dir(), "temp_hr_cache")
download_dir = download_nhdplushr(temp_dir, c( "0803","0804","0805","0806","0807"))

nhdflowline = get_nhdplushr(download_dir, file.path(download_dir, "nhdplus_out.gpkg"),
layers="NHDFlowline",
keep_cols = c("Permanent_Identifier", "REACHCODE", "StreamOrde", "Shape"))
`

[dblodgett-usgs]

I have a few fixes pending in #381 that will fix this. Once merged, install from github and you should be good to go @tjstagni

[tjstagni]

@dblodgett-usgs Just tried to run with the updated package and I have a new error. This issue likely applies to all of the basins mentioned above but I tested with these basins, and I have the following error message with running get_nhdplushr().

"1403" "1405" "1406" "1401" "1402"
Error in check_names(flowlines, "make_standalone_tonode") : Missing some required attributes in call to: make_standalone_tonode. Expected: COMID or NHDPlusHR equivalents.

I also still have the same issue mentioned above with download_nhdplushr() for the following basins
"0512"
Error in zip::unzip(out_file, exdir = out[length(out)]) : zip error: Cannot open zip file \AppData\Roaming\R\data\R\nhdplusTools\temp_hr_cache\05\NHDPLUS_H_0512_HU4_GDB.zip for reading in file zip.c:141

[dblodgett-usgs]

I see this -- it is only happening with th 140* ones. There are a bunch of name changes in these nhdplushr VPUs that are really confounding. I'll get a fix up shortly.

[dblodgett-usgs]

This was actually something different -- good bug to get dealt with!

[dblodgett-usgs]

latest on github should be good now and I am working on a CRAN release.

[tjstagni]

@dblodgett-usgs thank you so much for quickly working on these issues! The issue with get_nhdplushr() seems to be solved now. I still have the issue mentioned above with download_nhdplushr() for basin "0512"

Error in zip::unzip(out_file, exdir = out[length(out)]) : zip error: Cannot open zip file C:\Users\tstagnitta\AppData\Roaming\R\data\R\nhdplusTools\temp_hr_cache\05\NHDPLUS_H_0512_HU4_GDB.zip for reading in file zip.c:141

[tjstagni]

@dblodgett-usgs, Actually it seems like there still may be an issue with get_nhdplushr(), while the table will become available the "Permanent_Identifer" column is missing from the table. This is an issue with the other basins mentioned in the original issue at the beginning of this thread.

hr = get_nhdplushr(download_dir[j], file.path(download_dir[j], "nhdplus_out.gpkg"), layers="NHDFlowline", keep_cols = c("Permanent_Identifier", "REACHCODE", "StreamOrde", "Shape"))
example output

[dblodgett-usgs]

zip issue is a legit issue with zip::unzip. I'm putting in a try catch to try using utils::unzip as well. In some cases utils::unzip is a problem, but hopefully trying both will get us there!

[dblodgett-usgs]

If I run this:

library(nhdplusTools)
temp_dir = file.path(nhdplusTools_data_dir(), "temp_hr_cache")
download_dir = download_nhdplushr(temp_dir, c("0512"))

nhdflowline = get_nhdplushr(download_dir, file.path(download_dir, "nhdplus_out.gpkg"),
                            layers="NHDFlowline",
                            keep_cols = c("Permanent_Identifier", "REACHCODE", "StreamOrde", "Shape"))

in a clean session, I get:

> names(nhdflowline$NHDFlowline)
[1] "Permanent_Identifier" "REACHCODE"            "StreamOrde"           "Shape"    

Not sure what is going on there, but the current package code seams to be handling that correctly.

Zip handling should be better now. I'll leave this open till you've verified. Glad to get these issues ironed out. Thanks!

[tjstagni]

@dblodgett-usgs Thanks, the issue without having the Permanent_Identifier column is with the following basins

"1403" "1405" "1406" "1401" "1402" "1407" "1408"
"0202" "0203" "0204" "0205" "0206" "0207" "0208"
"0108" "0110" "0107"

[dblodgett-usgs]

OK, this one is super annoying.

Some of the NHDPlusHR VPUs have all lower case attributes and some have the mixed case that you are used to.

I'm going to do two things.

1. I'll add "Permanent_Identifier" to the list of attributes that gets normalized to mixed case convention.
2. I'll do the attribute filtering with a case insensitive filter so you at least get the data back when you ask for it.

[dblodgett-usgs]

OK -- give what I just merged a go. I think you should be ok now.

[tjstagni]

It seems to be working now, thanks again for working quickly on these updates!