Requested by @DaveEslinger in 10/1/2021 meeting.

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

The NoData flag is currently set to -9999. That value is well within the range of values expected in the comparison results. It might be best to use something different like -32768 (16-bit), or at least add another couple of 9s: -999999. That takes 17 bits.

Additional context

No response

Should the units of outputs be per year, like kg/year, L/year etc? This may tell more about the outputs of the results and provide context. @DaveEslinger @NSPECT thoughts?

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

Selecting pollutants requires users to edit a text field to specify Y or N for pollutants. However, the Align Rasters tool uses a checkbox gui for a similar function to pick layers to align.

Can that checkbox raster be used for pollutants?

Additional context

No response

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar issue do not exist

• I have searched existing issues to make sure a similar issue does not exist

What is the bug or the crash?

In the help, under Rainy Days section, "access" should be "excess"

Steps to reproduce the issue

1. Open Run Pollution Analysis Tool
2. Look at Number of Rainy Days in a Year in the help pane

Versions

QGIS version
3.22.3-Białowieża
QGIS code revision
1628765ec7
Qt version
5.15.2
Python version
3.9.5
GDAL/OGR version
3.4.1
PROJ version
8.2.1
EPSG Registry database version
v10.041 (2021-12-03)
GEOS version
3.10.0-CAPI-1.16.0
SQLite version
3.35.2
PDAL version
2.3.0
PostgreSQL client version
13.0
SpatiaLite version
5.0.1
QWT version
6.1.3
QScintilla2 version
2.11.5
OS version
Windows 10 Version 1909

Active Python plugins
curve_number_generator
1.3
mapswipetool_plugin
1.2
QNSPECT
0.1
db_manager
0.1.20
grassprovider
2.12.99
MetaSearch
0.3.5
processing
2.12.99
sagaprovider
2.12.99

Additional context

No response

When using a custom landcover, it would be good if the table of Pollutants shown included the values found in the CSV file that is loaded.

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar issues do not exist

• I have searched existing issues to make sure a similar issue does not exist

What is the bug or the crash?

Division by zero should be gracefully handled in comparison as well.

A strategy is needed to address division by zero for both direct and percent comparison. @DaveEslinger how should we handle them?

Steps to reproduce the issue

Run comparison tool.

Screenshots and Attachements

No response

Versions

3.22.4

Additional context

No response

It needs to be investigated that if LS Factor are always SFD, then does it make scientific sense to have MFD Flow accumulation?

If not, then the MFD option should be removed from Run Erosion Analysis tool until #45 is fixed.

Requested by Dave.

Group returned layers in a Layer Tree Group using Post Processor.

Useful links:
https://qgis.org/pyqgis/3.2/core/Processing/QgsProcessingLayerPostProcessorInterface.html?highlight=qgsprocessinglayerpostprocessorinterface

https://gis.stackexchange.com/a/401440/142232
https://gis.stackexchange.com/a/384996/142232

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar issues do not exist

• I have searched existing issues to make sure a similar issue does not exist

What is the bug or the crash?

When changing from mg to kg prior to accumulation, we didn't change the name of the accumulated file needed to calculate concentration.

Steps to reproduce the issue

Open Run Pollution Analysis
Run with any pollutant, and check the "Output Concetration Rasters"
Execute, will get failure with message similar to:
Traceback (most recent call last):
File "C:\Users/davee/AppData/Roaming/QGIS/QGIS3\profiles\dev/python/plugins\QNSPECT\algorithms\run_analysis\run_pollution_analysis.py", line 447, in processAlgorithm
"input_a": outputs[pol + "accum_mg"]["OUTPUT"],
KeyError: 'Leadaccum_mg'

Execution failed after 50.03 seconds

Screenshots and Attachements

No response

Versions

QGIS version
3.24.1-Tisler
QGIS code revision
5709b82461
Qt version
5.15.2
Python version
3.9.5
GDAL/OGR version
3.4.2
PROJ version
9.0.0
EPSG Registry database version
v10.054 (2022-02-13)
GEOS version
3.10.2-CAPI-1.16.0
SQLite version
3.38.1
PDAL version
2.3.0
PostgreSQL client version
unknown
SpatiaLite version
5.0.1
QWT version
6.1.3
QScintilla2 version
2.11.5
OS version
Windows 10 Version 2009

Active Python plugins
curve_number_generator
1.3
daves_proc
0.1
debug_vs
0.7
pluginbuilder3
3.2.1
plugin_reloader
0.9.1
valuetool
3.0.15
db_manager
0.1.20
grassprovider
2.12.99
MetaSearch
0.3.6
processing
2.12.99
sagaprovider
2.12.99
QNSPECT
0.1

Additional context

Bug fix is done, pull request coming

When making a modification to a pollutant run, the new results do not get loaded into a group. They just end up wherever the current focused layer is. This occurs even when changing the name of the run, which would be the name of the group, and when putting them in a new upper level directory. Closing the Run Pollutant window and reopening a new one, then running also gives this issue. I am only getting proper grouping with the first run in a QGIS session.

Note that they DO get put into the correct directory.

Below is the logfile result that seems to indicate where the error occurs. This error occurs when rerunning, but not on a new run.

Execution completed in 13.29 seconds
Results:
{'Runoff Accumulated': 'E:\GIS\QNSPECT\100x99x1\Feb21\Debug3_run_100cmx1rd\Runoff '
'Accumulated.tif',
'Runoff Local': 'E:\GIS\QNSPECT\100x99x1\Feb21\Debug3_run_100cmx1rd\Runoff '
'Local.tif',
'poll_1 Accumulated': 'E:\GIS\QNSPECT\100x99x1\Feb21\Debug3_run_100cmx1rd\poll_1 '
'Accumulated.tif',
'poll_1 Local': 'E:\GIS\QNSPECT\100x99x1\Feb21\Debug3_run_100cmx1rd\poll_1 '
'Local.tif'}

Loading resulting layers
Algorithm 'Run Pollution Analysis' finished

We need to create new SVGs for the banner and logo that use vectors for the letters, not a description of the fonts. Dave can do this.
DLE

The modify land use raster uses "Clipped extent" as the default name when using a temporary file. Can this be changed to "Modified Land Use" or something similar?

Just as it says..., tested twice

For example, the percent comparison here.

My first guess is, it is because of no data values, and edges.

One possible solution is to set the output value as no data value if any of the base rasters have no data value.

This error message might be related to that

This issue is to track GRASS no data value issue that is causing nodata value in accumulated rasters.

A ticket has been filed with GRASS
OSGeo/grass#2003

Possible Solutions

• run without -a flag and then add output with (local * 0) so as to remove nodata values, and then multiply -ve values with +ve

The output of the Align Rasters component can be temporary files handled by QGIS. Since this component is designed to create permanent files that will feed into other components, it might be better to require the output location to be a permanent path.

Current help wording has unnecessary link to other modules

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

When the Select File window opens for loading rasters, the filter is set to "All files". It would be nice to have it limited to common raster types. This may require a hard-coded option, since I don't see it in the default list shown in QGIS 3.20 and Win 10. The impact is shown in the attached screen shots.

Additional context

With default filter:

Just geotiffs:

This would display a user-friendly message, when a user tries to load these algorithms without having dependency i.e. GRASS

https://github.com/qgis/QGIS/blob/adc9d072cd5ef257b5fcf50eb9b86b179c782503/python/plugins/grassprovider/Grass7Algorithm.py#L1066

https://qgis.org/pyqgis/3.0/core/Processing/QgsProcessingAlgorithm.html?highlight=canexecute#qgis.core.QgsProcessingAlgorithm.canExecute

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar issue do not exist

• I have searched existing issues to make sure a similar issue does not exist

What is the bug or the crash?

Not a bug per se, but the Hydrologic Soils are spelled as Hydrographic Soils throughout the tools.

Steps to reproduce the issue

Noticeable in Run Erosion Analysis and Run Pollution Analysis tools, both for the raster dropdown and in the help. Possibly elsewhere too.

Versions

QGIS version
3.22.3-Białowieża
QGIS code revision
1628765ec7
Qt version
5.15.2
Python version
3.9.5
GDAL/OGR version
3.4.1
PROJ version
8.2.1
EPSG Registry database version
v10.041 (2021-12-03)
GEOS version
3.10.0-CAPI-1.16.0
SQLite version
3.35.2
PDAL version
2.3.0
PostgreSQL client version
13.0
SpatiaLite version
5.0.1
QWT version
6.1.3
QScintilla2 version
2.11.5
OS version
Windows 10 Version 1909

Active Python plugins
curve_number_generator
1.3
mapswipetool_plugin
1.2
QNSPECT
0.1
db_manager
0.1.20
grassprovider
2.12.99
MetaSearch
0.3.5
processing
2.12.99
sagaprovider
2.12.99

Additional context

No response

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

Land use and land cover are not synonymous. Both CCAP and NLCD are land cover data sets not land use. It is important to those who deal with such things.

Additional context

Patched by NSPECT in a fork. Pull request forthcoming.

Creating issue for tracking.

When overwriting the output file, this is what is happening in different QGIS versions and different use cases. In most cases, data is successfully processed despite messages. Although sometimes you have to refresh the map to see changes.

A ticket has been filed with QGIS
qgis/QGIS#47233

Possible ways:

Load Rasters with Style in QGIS Canvas

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

In the current setup, to ship/upload plugin, the whole repository is being zipped. The repo contains files and folders that are not required for a working plugin, for example, GitHub issue templates, readme, drawio diagrams, .gitignore etc.

QGIS will reject the plugin if it contains unnecessary stuff.

We should move the core plugin contents to a sub QNSPECT folder and zip and ship only that folder.

Additional context

No response

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar issue do not exist

• I have searched existing issues to make sure a similar issue does not exist

What is the bug or the crash?

QNSPECT users almost always are working within a defined area of interest such as a polygon watershed basin. Align Rasters currently clips by corner coordinates providing a rectangular output. If Clip Raster by Mask Layer could be used in Align Rasters the output would be based on the shape of the area of interest.

Steps to reproduce the issue

Go to Align Rasters
Add reference Raster and Rasters to Align
Add a polygon vector file as the Clipping Extent

Versions

QGIS version
3.22.3-Białowieża

Additional context

No response

Reference QGIS and GDAL issue on respective repos for background on this.

Overwrite operations (3.18 above) and Update (3.16) operations are failing for GDAL raster calc.

To temporarily overcome the issue, pass an explicit --overwrite flag to GDAL operations, so that it overwrites not update files and fails if it is loaded in the canvas.

The runoff calculations here cause runtime errors.

https://github.com/Dewberry/QNSPECT/blob/fbff1a01a10e5e40d58d71b3c8c8302b032b6cbe/algorithms/run_analysis/Runoff_Volume.py#L138

<string>:1: RuntimeWarning: overflow encountered in square

This might be because of #29

• Checked that it is not caused by misaligned cells

Some of the local pollutants dont show the units of their color scale (i.e., they don't have (mg) or (L)) when the Run Pollutant Analysis tool is run after loading it with the Load previous Analysis tool. Very odd.

I've tried to attach a screen grab here. The first group was created using the Run P Analysis Gui directly. The second was after closing that GUI, then using the Load Previous funtion to reload that GUI. Note that the Zinc and TSS local results NEVER had their units.

Now that Pollution Analysis is grouping its results, can we get other functions to do the same? The name for the Comparison tool grouping could be the Folder1 name vs folder2 name.

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar issue do not exist

• I have searched existing issues to make sure a similar issue does not exist

What is the bug or the crash?

Null values in LC lookup causes the tool to fail. Perhaps include some language in the tool help and documentation that each cell needs a value

Steps to reproduce the issue

1. Create a modified LC lookup table with NULL values in place of valid pollutant coefficients
2. Run the Pollution Analysis tool

Versions

QGIS version
3.22.3-Białowieża
QGIS code revision
1628765ec7
Qt version
5.15.2
Python version
3.9.5
GDAL/OGR version
3.4.1
PROJ version
8.2.1
EPSG Registry database version
v10.041 (2021-12-03)
GEOS version
3.10.0-CAPI-1.16.0
SQLite version
3.35.2
PDAL version
2.3.0
PostgreSQL client version
13.0
SpatiaLite version
5.0.1
QWT version
6.1.3
QScintilla2 version
2.11.5
OS version
Windows 10 Version 1909

Active Python plugins
curve_number_generator
1.3
mapswipetool_plugin
1.2
QNSPECT
0.1
db_manager
0.1.20
grassprovider
2.12.99
MetaSearch
0.3.5
processing
2.12.99
sagaprovider
2.12.99

Additional context

No response

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

Let’s change the name for the “Desired Outputs [optional]” group in the Run Pollution Analysis tool to something like “Pollutant Outputs [optional]”. This is really what people are looking to generate, and I totally missed it the first few times trying to run the tool. If possible, it would be good to make it very clear what this is, and that it must be selected in order to generate outputs.

Additional context

No response

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

Somehow we completely missed the functionality to only run over a selected subset of watersheds/catchments in an Area of Interest. This was initially done to save run time. The current tool is even slower, so it may be worth investigating.

Additional context

No response

Runoff calculation is calling GDAL raster calc and overwriting an input raster.

Happening at

https://github.com/Dewberry/QNSPECT/blob/fbff1a01a10e5e40d58d71b3c8c8302b032b6cbe/algorithms/run_analysis/Runoff_Volume.py#L153

gdal_calc.bat --calc "(B!=0) * A" --format GTiff --type Float32 --NoDataValue -9999.0 -A "C:\\Users\\asiddiqui\\Desktop\\plugin_test10\\Runoff Local.tif" --A_band 1 -B C:/Users/asiddiqui/AppData/Local/Temp/processing_JLESJk/c7f634307857454293e4dcb816f22855/OUTPUT.tif --B_band 1 --outfile "C:\\Users\\asiddiqui\\Desktop\\plugin_test10\\Runoff Local.tif"
GDAL command output:
ERROR 1: C:\\Users\\asiddiqui\\Desktop\\plugin_test10\\Runoff Local.tif, band 1: Write operation not permitted on dataset opened in read-only mode

0.. Block writing failed

Results: {'OUTPUT': 'C:\\Users\\asiddiqui\\Desktop\\plugin_test10\\Runoff Local.tif'}

The solution is to save the temporary runoff local to a temp file rather than file destination at

https://github.com/Dewberry/QNSPECT/blob/fbff1a01a10e5e40d58d71b3c8c8302b032b6cbe/algorithms/run_analysis/Runoff_Volume.py#L142

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar issues do not exist

• I have searched existing issues to make sure a similar issue does not exist

What is the bug or the crash?

Lets say we have raster A = [[5,1]] and B = [[1,2]]

When we calculate percent difference here expression="100 * ((A - B) / A)"

Our result will be [[80%, -100%]]. How do we interpret this ?

1. A is 80% of B, and A is -100% of B?
2. B is 80% smaller than A, B is -100% smaller than A, which means B is 100% greater than A?

How to interpret results?

Steps to reproduce the issue

N/A

Screenshots and Attachements

No response

Versions

3.22.4

Additional context

No response

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

If inputs are temporary layers, they will not be available the next time load previous run will be used on the configuration file. Hence a warning should be raised for users if inputs are temp layers.

Additional context

No response

Currently, GRASS does not support multi-flow direction calculations for the LS-Factor raster. For now the tool explicitly uses single-flow direction for its calculation. If GRASS updates their calculations or a workaround is found for this, the will need to be updated to have the user's MFD input be represented in in the LS-Factor calculation.

The output rasters in Align Rasters inherit their names from the names of the input file names. If two or more input rasters share the same name, they will overwrite each other.

Currently, it is an intermediate raster and is not returned to the user, but one may argue that it is a useful thing to report back to the user since it tells about Soil loss (tonnes/year)

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

Given our current setup, it is possible to reclass from CCAP to NLCD and vice-versa. That doesn't make sense and should not be permitted.

I suggest adding a check on range of values in the incoming raster to determine type and then ask.

Additional context

No response

If you perform a run of the erosion module, then look at the top cell of a watershed, you will notice that both local and accumulated values are the same. This is not correct, since local should be in Kilograms and accumulated should be in Megagrams. One of these is off by a factor of 1000. I suspect it is the Accumulated value, but haven't worked out all the calculations by hand yet.

When using Modify Land Use to add a new value to a raster, if the new value is outside the range of the original raster (e.g., '30' is added to a raster with values between '0' and '29'), the new value will not be displayed by default. This bug is present in QGIS when running the "Rasterize (Overwrite With Attribute)" tool.

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

Tests are important. We don't have any tests right now. Some unit tests and e2e tests should be added at the earliest chance.

Additional context

No response

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar issues do not exist

• I have searched existing issues to make sure a similar issue does not exist

What is the bug or the crash?

Output from Analysis routines do not consistently load the correct color tables. Typical error warnings seem to be:
"2022-03-22T10:46:13 CRITICAL ERROR 6: C:\Users\davee\AppData\Local\Temp\processing_pVfCXl\2a143c83a76741c4953606acf6b98cdf\accumulation.tif, band 1: SetColorTable() only supported for Byte or UInt16 bands in TIFF format." and

2022-03-22T10:46:13 CRITICAL WARNING: Too many values, color table cut to 65535 entries

I don't these are related, but they seem to be what show up. I say that because in back-to-back runs, the first run (with all18 possible pollutant output raster) only 1 raster was colored, in the second one all were colored correctly.

Steps to reproduce the issue

Run Pollutant analysis. See if all rasters get consistent (non-QGIS gray) color ramps. Try it again. It seems arbitrary.

Screenshots and Attachements

Correct colors mapping, all the same nutrients get the same color ramp, i.e., all Lead rasters are red, so only one of each is shown:

No colors loaded, default gray colors:

Versions

QGIS version
3.20.3-Odense
QGIS code revision
495fbaec
Qt version
5.15.2
Python version
3.9.5
GDAL/OGR version
3.3.1
PROJ version
8.1.1
EPSG Registry database version
v10.018 (2021-04-02)
GEOS version
3.9.1-CAPI-1.14.2
SQLite version
3.35.2
PDAL version
2.3.0
PostgreSQL client version
13.0
SpatiaLite version
5.0.1
QWT version
6.1.3
QScintilla2 version
2.11.5
OS version
Windows 10 Version 2009

Active Python plugins
curve_number_generator
daves_proc
debug_vs
pluginbuilder3
plugin_reloader
QNSPECT
valuetool
db_manager
MetaSearch
processing

Additional context

We've talked about this, but I don't think it is documented as an issue.

Latest QNSPECT version

• I am running the latest QNSPECT version

Similar requests do not exist

• I have searched existing issues to make sure a similar request does not exist

Feature description

It would be nice, and time saver, if it were possible to snap the output soils rasters to a reference raster when converting the soils from polygon to raster.

Additional context

No response

The following error is being reported:

C:\OSGeo4W\apps\Python39\lib\site-packages\GDAL-3.4.0-py3.9-win-amd64.egg\osgeo_utils\gdal_calc.py:518: RuntimeWarning: invalid value encountered in multiply 
myResult = ((1 * (myNDVs == 0)) * myResult) + (myOutNDV * myNDVs)

This might be related to #39