dustming / basinmaker Goto Github PK
View Code? Open in Web Editor NEWAn automated GIS toolbox for watershed delineation with lakes
License: Artistic License 2.0
basinmaker's People
Contributors
Stargazers
Watchers
basinmaker's Issues
QGIS Installation Error
I ran into this error when running the full installation script
ERROR: Unable to make mapset element
cell_misc/catchment_pourpoints_outside_lake
(./output_full_installation\grassdb\main_working_location\PERMANENT/cell_misc/catchment_pourpoints_outside_lake):
No such file or directory
ERROR: Raster map <catchment_pourpoints_outside_lake> not found
Module run None r.null map=catchment_pourpoints_outside_lake setnull=-9999,0 ended with error
Process ended with non-zero return code 1. See errors in the (error) output.
please report error message to [email protected] or create an issue in https://github.com/dustming/basinmaker/issues
Comment characters after reservoir name in rvh file
Minor point, but I would recommend that the extra comment characters do not appear after the reservoir name. Best to avoid comments at the end of Raven line files where possible. E.g.
:Reservoir Lake_107979
Instead of:
:Reservoir Lake_107979 ###
Grass 82 error
Hi Ming,
Using GRASS 8.2.1 (current latest version) gives an error message when trying to delineate a watershed from a DEM.
Here are the error messages:
Cannot create: /root/BasinMaker/bm_tmp/grassdb/main_working_location/PERMANENT with the following options: . Executing:
grass -c -e /root/BasinMaker/bm_tmp/grassdb/main_working_location/PERMANENT
GRASS said:
"Starting GRASS GIS...\nERROR: Mapset </root/BasinMaker/bm_tmp/grassdb/main_working_location/PERMANENT> already exists.\nExiting..."
Even if I delete the "bm_tmp" folder, the error comes back. I think it happens since GRASS 8.0, but I'm not 100% sure. The BasinMaker version is 3.0.3. Let me know if you need more details!
Writing lakes.rvh file termination
The lakes.rvh file is written out with a new line at the end as:
###New Lake starts
Recommend that this is not written at the end of the file, although as a comment it does not impact the function of the file.
Generate_HRUs cannot add the field a_HRULake_ID
When running the Generate_HRUs function, it says it cannot add the field "a_HRULake_ID". I think it may be because the field name is longer than 10 characters, but it may be something else too.
Here are the messages from the script:
ERROR 6: Failed to add field named 'a_HRULake_ID'
ERROR: Unable to create column <a_HRULake_ID>
Module run None v.out.ogr --o input=union_1 output=/root/BasinMaker/OIH_Output/network_after_filter_lakes/union_1_.shp format=ESRI_Shapefile ended with error
Process ended with non-zero return code 1. See errors in the (error) output.
The version of BasinMaker is 2.2.6. Let me know if I forgot important info!
Incompatibility between the Decrease_River_Network_Resolution() post-processing tool and MacOS
There is an issue with the Decrease_River_Network_Resolution() post-processing tool, which I think is specific to MacOS systems.
When a shapefile is opened (for example in QGIS) and perhaps modified on a mac, hidden files, prefixed with ._, are often created for the corresponding shapefile. The folder where the original shapefile was located may now contain a bunch of hidden files - for example:
sl_connected_lake.cpg
._sl_connected_lake.dbf
sl_connected_lake.dbf
sl_connected_lake.prj
._sl_connected_lake.shp
sl_connected_lake.shp
._sl_connected_lake.shx
sl_connected_lake.shx
When running the basinmaker.postprocess().Decrease_River_Network_Resolution() post-processing tool, the simplify_routing_structure_by_drainage_area_purepy function in script increasedapurepy.py will try to find the shapefiles it needs in the provided Routing_Product_Folder. The way it looks for them can cause issues when these hidden files are present. On L106 of increasedapurepy.py the script looks for all files in that folder ending with ".shp", which will lead to the following error:
File "fiona/ogrext.pyx", line 136, in fiona.ogrext.gdal_open_vector
File "fiona/_err.pyx", line 291, in fiona._err.exc_wrap_pointer
fiona._err.CPLE_OpenFailedError: '._sl_connected_lake.shp' not recognized as a supported file format.
I think that excluding the files starting with ._ from the search should do the trick.
Subbasins intersecting lakes after postprocessing NALRP v2.1
NALRP v2.1 for the Mackenzie basin (drainage_region_0002_v2-1) was postprocessed as following:
bm.Remove_Small_Lakes(
connected_lake_area_thresthold=20,
non_connected_lake_area_thresthold=20,
selected_lake_ids=[],
gis_platform="purepy",
)
bm.Decrease_River_Network_Resolution(
minimum_subbasin_drainage_area=500,
gis_platform="purepy",
)
Output while processing:
Obtain selected Lake IDs done
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
111450 2098024 2097973 0.000102 0.0 0.134715 131.661119 4.584521e+06 ... 0.0 0.0 None None -134.956238 68.060896 POLYGON ((-134.93000 68.08208, -134.93000 68.0...
111502 2097972 2098021 0.000102 0.0 0.061054 134.707863 1.068796e+06 ... 0.0 0.0 None None -134.982126 68.073958 POLYGON ((-134.99250 68.08292, -134.99083 68.0...
[2 rows x 33 columns]
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
111450 2098024 2097973 0.000102 0.0 0.134715 131.661119 4.584521e+06 ... 0.0 0.0 None None -134.956238 68.060896 POLYGON ((-134.93000 68.08208, -134.93000 68.0...
111502 2097972 2098021 0.000102 0.0 0.061054 134.707863 1.068796e+06 ... 0.0 0.0 None None -134.982126 68.073958 POLYGON ((-134.99250 68.08292, -134.99083 68.0...
[2 rows x 33 columns]
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
147874 2099455 2099450 0.000274 0.0 0.166309 145.311279 1.523418e+05 ... 0.0 0.0 None None -135.141024 68.469149 MULTIPOLYGON (((-135.13750 68.46542, -135.1375...
147876 2099453 2099455 0.000274 0.0 0.141743 182.145558 1.187094e+06 ... 0.0 0.0 None None -135.149051 68.467268 POLYGON ((-135.15333 68.47875, -135.15333 68.4...
[2 rows x 33 columns]
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
147874 2099455 2099450 0.000274 0.0 0.166309 145.311279 1.523418e+05 ... 0.0 0.0 None None -135.141024 68.469149 MULTIPOLYGON (((-135.13750 68.46542, -135.1375...
147876 2099453 2099455 0.000274 0.0 0.141743 182.145558 1.187094e+06 ... 0.0 0.0 None None -135.149051 68.467268 POLYGON ((-135.15333 68.47875, -135.15333 68.4...
[2 rows x 33 columns]
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
152082 2103421 2103419 0.000001 0.0 0.384312 136.198539 1.836777e+05 ... 0.0 0.0 None None -133.766425 68.120950 POLYGON ((-133.77250 68.12208, -133.77250 68.1...
152083 2103422 2103419 0.000001 0.0 0.350262 190.493629 1.005215e+06 ... 0.0 0.0 None None -133.769394 68.125235 POLYGON ((-133.78667 68.12542, -133.78667 68.1...
[2 rows x 33 columns]
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
111813 2098651 2098611 0.000928 0.0 0.160943 150.262882 898856.665678 ... 0.0 0.0 None None -134.909733 68.532048 POLYGON ((-134.92167 68.53375, -134.92167 68.5...
111892 2098606 2098922 0.001286 0.0 0.096005 180.838428 670771.497977 ... 0.0 0.0 None None -134.908498 68.538958 POLYGON ((-134.90917 68.54208, -134.90917 68.5...
[2 rows x 33 columns]
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
111813 2098651 2098611 0.000928 0.0 0.160943 150.262882 898856.665678 ... 0.0 0.0 None None -134.909733 68.532048 POLYGON ((-134.92167 68.53375, -134.92167 68.5...
111892 2098606 2098922 0.001286 0.0 0.096005 180.838428 670771.497977 ... 0.0 0.0 None None -134.908498 68.538958 POLYGON ((-134.90917 68.54208, -134.90917 68.5...
[2 rows x 33 columns]
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
59705 2142714 2142722 0.000086 0.0 0.082716 186.903686 321878.799365 ... 0.0 0.0 None None -134.831006 67.913897 POLYGON ((-134.83833 67.91375, -134.83833 67.9...
112893 2097564 2098594 0.018114 0.0 0.079473 180.261746 321878.799365 ... 0.0 0.0 None None -134.831006 67.913897 POLYGON ((-134.83833 67.91375, -134.83833 67.9...
[2 rows x 33 columns]
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
59705 2142714 2142722 0.000086 0.0 0.082716 186.903686 321878.799365 ... 0.0 0.0 None None -134.831006 67.913897 POLYGON ((-134.83833 67.91375, -134.83833 67.9...
112893 2097564 2098594 0.018114 0.0 0.079473 180.261746 321878.799365 ... 0.0 0.0 None None -134.831006 67.913897 POLYGON ((-134.83833 67.91375, -134.83833 67.9...
[2 rows x 33 columns]
It is not a non connected lake catchment
SubId DowSubId RivSlope RivLength BasSlope BasAspect BasArea ... DA_Obs DA_error Obs_NM SRC_obs centroid_x centroid_y geometry
152082 2103421 2103419 0.000001 0.0 0.384312 136.198539 1.836777e+05 ... 0.0 0.0 None None -133.766425 68.120950 POLYGON ((-133.77250 68.12208, -133.77250 68.1...
152083 2103422 2103419 0.000001 0.0 0.350262 190.493629 1.005215e+06 ... 0.0 0.0 None None -133.769394 68.125235 POLYGON ((-133.78667 68.12542, -133.78667 68.1...
[2 rows x 33 columns]
.../basinmaker/func/purepy.py:323: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.
tost[coly] = tost.geometry.centroid.y
.../basinmaker/func/purepy.py:324: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.
tost[colx] = tost.geometry.centroid.x
.../basinmaker/func/purepy.py:323: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.
tost[coly] = tost.geometry.centroid.y
.../basinmaker/func/purepy.py:324: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.
tost[colx] = tost.geometry.centroid.x
.../basinmaker/func/pdtable.py:2104: FutureWarning: The provided callable <built-in function max> is currently using SeriesGroupBy.max. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "max" instead.
["HyLakeId"])["DrainArea"].transform(max)
.../basinmaker/func/purepy.py:323: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.
tost[coly] = tost.geometry.centroid.y
.../basinmaker/func/purepy.py:324: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.
tost[colx] = tost.geometry.centroid.x
.../basinmaker/func/purepy.py:323: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.
tost[coly] = tost.geometry.centroid.y
.../basinmaker/func/purepy.py:324: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.
tost[colx] = tost.geometry.centroid.x
In the output after decreasing river network resolution, lakes (blue) can be observed crossing subbasins boundaries (green):
Python 3.10.12
Package Version
affine 2.4.0
anyio 4.0.0
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asttokens 2.4.1
async-lru 2.0.4
attrs 23.1.0
Babel 2.13.1
basinmaker 3.1.0
beautifulsoup4 4.12.2
bleach 6.1.0
cdsapi 0.6.1
certifi 2023.5.7
cffi 1.16.0
cftime 1.6.3
charset-normalizer 3.1.0
click 8.1.7
click-plugins 1.1.1
cligj 0.7.2
comm 0.2.0
debugpy 1.8.0
decorator 5.1.1
defusedxml 0.7.1
exceptiongroup 1.1.3
executing 2.0.1
fastjsonschema 2.19.0
filelock 3.13.1
fiona 1.9.5
fqdn 1.5.1
GDAL 3.4.1
gdown 4.7.1
geopandas 0.14.1
idna 3.4
iniconfig 2.0.0
ipykernel 6.26.0
ipython 8.17.2
ipywidgets 8.1.1
isoduration 20.11.0
jedi 0.19.1
Jinja2 3.1.2
joblib 1.3.2
json5 0.9.14
jsonpointer 2.4
jsonschema 4.19.2
jsonschema-specifications 2023.11.1
jupyter 1.0.0
jupyter_client 8.6.0
jupyter-console 6.6.3
jupyter_core 5.5.0
jupyter-events 0.9.0
jupyter-lsp 2.2.0
jupyter_server 2.10.0
jupyter_server_terminals 0.4.4
jupyterlab 4.0.8
jupyterlab-pygments 0.2.2
jupyterlab_server 2.25.1
jupyterlab-widgets 3.0.9
MarkupSafe 2.1.3
matplotlib-inline 0.1.6
mistune 3.0.2
nbclient 0.9.0
nbconvert 7.11.0
nbformat 5.9.2
nest-asyncio 1.5.8
netCDF4 1.6.5
notebook 7.0.6
notebook_shim 0.2.3
numpy 1.26.2
overrides 7.4.0
packaging 23.2
pandas 2.1.3
pandocfilters 1.5.0
parso 0.8.3
pexpect 4.8.0
pip 23.3.1
platformdirs 4.0.0
pluggy 1.3.0
prometheus-client 0.18.0
prompt-toolkit 3.0.41
psutil 5.9.6
ptyprocess 0.7.0
pure-eval 0.2.2
pycparser 2.21
Pygments 2.16.1
pyparsing 3.1.1
pyproj 3.6.1
PySocks 1.7.1
pytest 7.4.3
python-dateutil 2.8.2
python-json-logger 2.0.7
pytz 2023.3.post1
PyYAML 6.0.1
pyzmq 25.1.1
qtconsole 5.5.0
QtPy 2.4.1
rasterio 1.3.9
rasterstats 0.19.0
referencing 0.31.0
requests 2.31.0
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rpds-py 0.12.0
scipy 1.11.3
Send2Trash 1.8.2
setuptools 67.7.2
shapely 2.0.2
simpledbf 0.2.6
simplejson 3.19.2
six 1.16.0
sniffio 1.3.0
snuggs 1.4.7
soupsieve 2.5
stack-data 0.6.3
terminado 0.18.0
tinycss2 1.2.1
tomli 2.0.1
tornado 6.3.3
tqdm 4.65.0
traitlets 5.13.0
types-python-dateutil 2.8.19.14
typing_extensions 4.8.0
tzdata 2023.3
uri-template 1.3.0
urllib3 2.0.2
wcwidth 0.2.10
webcolors 1.13
webencodings 0.5.1
websocket-client 1.6.4
wget 3.2
wheel 0.40.0
widgetsnbextension 4.0.9
Function request - combine two or more sets of routing products in post-processing
Would be great to have a function to combine routing products from two or more folders into one product. This would be useful in instances where you have two sections of catchments that you ultimately want in one model, but with two distinct outlets.
Something like:
basinmaker.basinmaker.delineate.Combine_Routing_Products(routing_product_folders, gis_platform='qgis')
routing_product_folders = ['folder1','folder2']
gis_platform='qgis'
Output is a single routing product determined from combining one or more routing products. Key challenge would be to confirm that the individual routing products have non-overlapping extents. Note that this can currently be done manually with GIS software, and subsequent operations seem to work ok.
HRU ID 0 in shapefile but not rvh
The hru shapefiles (final_cat_hru) appears to be generated with an hru of ID zero (0). However, this element is not written to the rvh file. Since Raven requires all subbasin and HRU IDs to be positive integers (i.e. >0), ideally the shapefile generation assigns all positive integer IDs to avoid this issue.
install dependent packages line not working
There appears to be a minor error in the "install dependent packages" line on the installation instructions website (https://basinmaker.readthedocs.io/en/latest/installation.html#light-installation). Very minor but i did not realize this line did not work and had errors later in the installation process that I had to trace back to this issue.
python - m pip install pandas pytest scipy simpledbf netCDF4 jupyter
should be
python -m pip install pandas pytest scipy simpledbf netCDF4 jupyter
Error on installation
Might not be a great issue for github here, but I've failed on installation for full version of BasinMaker.
Worked through the installation procedure on https://basinmaker.readthedocs.io/en/latest/installation.html for full installation.
Everything went smooth, the only subtle difference I encountered was that my python installation location was in:
C:\OSGeo4W\bin\python.exe
and not in:
C:\OSGeo4W\apps\Python37\python.exe
as described in the documentation.
This was what I encountered upon attempting installation test:
C:\Temp\test>python test_full_installation.py
PyTables is not installed. No support for HDF output.
SQLalchemy is not installed. No support for SQL output.
Mask Region: Using provided DEM :
Logged warning: Duplicate provider native registered
[WinError 5] Access is denied
please report error message to [email protected] or create an issue in https://github.com/dustming/basinmaker/issues
Cheers,
Jeremy
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.