kalab-oto / rndop Goto Github PK

search by month in season

Numbers of filtered records in web app is differs from that returned by ndop_download using both spcies and families.

Bug reported by mail:

Kolik je v databázi : Kolik se stáhne při stáhnutí daného druhu : Kolik se stáhne, pokud stáhneš čeleď a pak filtruješ
Coenonympha pamphilus: 31195 : 31627 : 31734

Když vyfiltruješ daný druh po stáhnutí celé čeledi
Thymelicus lineola → 20978: Hesperiidae → 13268
Polyommatus icarus → 28308: Lycaenidae → 29826
Plebejus argus → 7407: Lycaenidae → 7747
Melitaea athalia → 9726: Nymphalidae → 9796
Melanargia galathea → 20978: Nymphalidae → 22874
Maniola jurtina → 33724: Nymphalidae → 22874
Coenonympha arcania → 6360: Nymphalidae → 6469
Coenonympha glycerion → 9925: Nymphalidae → 9974

search by date

Let the user know about crashes on the web filter side with printing message from webside (filter_page)

e.g.
<tr><td colspan="2"><ul style="color: red;"><li>Filtr vrací příliš mnoho výsledků, upřesněte prosím podmínky.</li></ul></td></tr> <tr>

separate function for table download get_csv_table.R - same way as get_locations.R

move table_payload list to ndop_search() function

Clean up the ndop_download() funciton after #8 and #9, and think of a way to run iterative search_payload, ndop_search and table download (for loop). This will allow more options for #10 and #2.

Test if ndop_search and table download can be iterated separately, with ndop_search returning list of tokens ndtoken. This helps with counting total number of occurrences in iterations.

Add search by families and groups to ndop_download.R

Setup user search payload option, that give user full control to set own raw search payload from provided template.

Define argument in ndop_downlad.R to pass sf polygon object, than reformat geometry to search payload.

Check the usability of the lookup tables that will be available in the nearby future.

write test for ndop_download and execute with github actions.

Bug reported by mail

Some species can't be downloaded while can be filtered, and returns occurrences in web app

Thymelicus lineola
Polyommatus icarus
Plebejus argus
Melitaea athalia
Melanargia galathea
Maniola jurtina
Coenonympha arcania
Coenonympha glycerion 

Error in vector("list", length = tables_num) : vector size cannot be NA/NaN

search by occurrence id

enable pass list of species to taxon argument in ndop_download function.

ndop_search() returns NA records

Hidden coordinates (<i>Skrytá lokalizace</i>) break the data type in the X and Y fields, fields in merged table results in strings with different decimal separator.

The raw table from NDOP should be translated and modified (e.g, DATUM_OD). This will probably follow the Darwin Core standard.

The raw data will be still available with ndop_download argument raw = True, so this argument will need to be added in established processes.

If the login is once stored in .Renviron it can't be changed with isop_login(). This disable to enter new user, or new password if changed.
Possible fix: add argument which deletes the username and password from .Renviron and promt for new credentials

The first request after login returns empty results. Fix this workaround. Try add checking the page in isop_login() function.

Bug reported by mail:

If promted string (e.g. species name) are in multiple values in database, and one of them has 0, none of them are downloaded (counts as 0 records). Similar case as #4.

Processing 2 species: Plebejus argus, Plebejus argus/argyrognomon,NA records found

As we know, when you want to download more species in one query you do something like this:

species <- ndop_download("Lutra lutra || Glis glis" ) That is great, but trouble kicks in, when you have to download for example 50 species and one species has 0 records. The process failed and you have no idea what species specifically has 0 records.

So what I came with this pretty easy solution (maybe not the fastest one):

species <- c(Lutra lutra" , "Glis glis")  #vector of species names

final_table <- data.frame()  #empty data frame to catch result

for (i in species){
  print(i) #print ongoing species
  result <- ndop_download(i) 
  final_table <- rbind(final_table , result ) #row bind restult
}