ciat-dapa / subsets_genebank_accessions Goto Github PK

Hi everyone.
This issue were created because when I run the script to get the accessions data from a crop in some cases the script generates a issue (in the image attached). The cases generates the issue are those cases I put like input the crops are in the crop table in the issue #40 in the column "Stored in the database" with value "Processing"

Thanks so much

Regards.

Create a web API to the tool that allows the communication between the indicators database and website.

• The web API must allow to query each collection present in the indicators database though a URI or endpoint that be identified easy for the collection to query. i.e. if I need to query the data corresponding to accessions, the URI must contain in your endpoint the identifier .../accessions/

Description

As an administrator user, I want to be able build a set of biotics indicators for each accession, with which latterly the users will may to create GenBank accessions subsets from filters based in them.

Acceptance criteria

Stage 1 - Pest resistance: To create the pest resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators pest resistance and latterly him will run the script that build them.                                                      

Stage 2 - Disease tolerance: To create the disease tolerance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators disease tolerance and latterly him will run the script that build them.                                                                                                                                                  
Stage 3 - Seed color:  To create the seed color indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators seed color and latterly him will run the script that build them.                                                                                                              

Stage 4 - Growth habit: To create the growth habit indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators growth habit and latterly him will run the script that build them.                           

Stage 5 - Root color: To create the root color indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators root color and latterly him will run the script that build them.                                                                                                              

Stage 6 - FOC-TR4 resistance: To create the FOC-TR4 resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators FOC-TR4 resistance and latterly him will run the script that build them.                  

Stage 7 - Black sigatoka resistance: To create the black sigatoka resistance indicator for each accession the administrator user must have a file,  once him had filled out it, the administrator user will proceed to select it in the section biotic indicators black sigatoka resistance and latterly him will run the script that build them.
                                                                                                                                                                                          
Stage 8 - Rust resistance: To create the rust resistance indicator for each accession the administrator user must have a file,  once him had filled out it, the administrator user will proceed to select it in the section biotic indicators rust resistance and latterly him will run the script that build them.
                                                                                                                                                                                          
Stage 9 - Wheat blast resistance:  To create the wheat blast resistance indicator for each accession the administrator user must have a file,  once him had filled out it, the administrator user will proceed to select it in the section biotic indicators wheat blast resistance and latterly him will run the script that build them.
                                                                                                                                                                                          
Stage 10 - Tar spot resistance:  To create the tar spot resistance indicator for each accession the administrator user must have a file,  once him had filled out it, the administrator user will proceed to select it in the section biotic indicators  tar spot resistance and latterly him will run the script that build them.
                                                                                                                                                                                         
Stage 11 - Septoria tritici resistance:  To create the Septoria tritici resistance indicator for each accession the administrator user must have a file,  once him had filled out it, the administrator user will proceed to select it in the section biotic indicators Septoria tritici resistance and latterly him will run the script that build them.
                                                                                                                                                                                          
Stage 12 - Leaf blight resistance:  To create the leaf blight resistance indicator for each accession the administrator user must have a file,  once him had filled out it, the administrator user will proceed to select it in the section biotic indicators leaf blight resistance and latterly him will run the script that build them.
                                                                                                                                                                                         
Stage 13 - SPFMV resistance: To create the SPFMV resistance indicator for each accession the administrator user must have a file,  once him had filled out it, the administrator user will proceed to select it in the section biotic indicators SPFMV resistance and latterly him will run the script that build them.
                                                                                                                                                                                         
Stage 14 - Begomovirus resistance:  To create the begomovirus resistance indicator for each accession the administrator user must have a file,  once him had filled out it, the administrator user will proceed to select it in the section biotic indicators begomovirus resistance and latterly him will run the script that build them.                                                                                                                                                                                          

Stage 15 - Blast resistance: To create the blast resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators blast resistance and latterly him will run the script that build them.
                                                                                                                                                                                         
Stage 16 - BLB resistance: To create the BLB resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators BLB resistance and latterly him will run the script that build them.
                                                                                                                                                                                         
Stage 17 - RYMV resistance: To create the RYMV resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators RYMV resistance and latterly him will run the script that build them.                                                                                                                                                                                          
Stage 18 - Leaf rust resistance:  To create the leaf rust resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators leaf rust resistance and latterly him will run the script that build them.                                                                                                                                                                                          }
Stage19 - Yellow rust resistance: To create the yellow rust resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators yellow rust resistance and latterly him will run the script that build them.                                                                                                                                                                                          
Stage 20 - Stem rust resistance: To create the stem rust resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators stem rust resistance and latterly him will run the script that build them.                                                                                                                         
Stage 21 - Powdery mildew resistance: To create the powdery mildew resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators powdery mildew resistance and latterly him will run the script that build them.
                                                                                                                                                                                          
Stage 22 - Net blotch resistance: To create the net blotch resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators net blotch resistance and latterly him will run the script that build them.                                                                                                                                                                                          
Stage 23 - Ascochyta blight resistance: To create the ascochyta blight resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators ascochyta blight resistance and latterly him will run the script that build them.                                                                                                                                                                               
Stage 24 - Orobanche resistance: To create the orobanche resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators orobanche resistance and latterly him will run the script that build them.                                                                                                                                                                                          
Stage 25 - Leaf miner resistance: To create the leaf miner resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators leaf miner resistance and latterly him will run the script that build them.                                                                                                                                                                                          
Stage 26 - Wilf resistance: To create the wilf resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators wilf resistance and latterly him will run the script that build them.                                                                                                                                                                                          
Stage 27 - Downy mildew resistance: To create the downy mildew resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators downy mildew resistance and latterly him will run the script that build them.                                                                                                                                                                                          
Stage 28 - Shoot fly resistance: To create the shoot fly resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators shoot fly resistance and latterly him will run the script that build them.                                                                                                                                                                                          
Stage 29 - Stem borer resistance: To create the stem borer resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators stem borer resistance and latterly him will run the script that build them.                                                                                                                                                                                                    
Stage 30 - Pod borer resistance: To create the pod borer resistance indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section biotic indicators pod borer resistance and latterly him will run the script that build them.**                                                                                                                         
Stage 31 - Successful operation: For each stage, if the operation flow is successful the system shows a message indicating it.                                                                                                                                                          

Stage 32 - Failed operation: For each stage, if the file structure isn't correctly, the system will show an exception with the message: "The file structure isn't correct."                                                             

Stage 33 - Missing value: For each stage, if the file has a column without value, the indicator for this accession won't create.                                                                                                                                                                                                                                                                                    
Stage 34 - Generate logs file:  For each stage the system will generates a log file, which is described in the HUS007.

Description

As an user in the web module, I want to be able limit the amount outcomes (accessions) that I want when creating a subset, to customize it according to my needs.

Acceptance criteria

1. The amounts available should be: < 25 accessions, between 25 and 50 accessions, between 50 and 100 accessions and > 100 accessions.

Custom indicator is the component which creates indicators for custom data that user can upload through the system. These indicators are not stored into the indicator database, because they are used just to create temporal subsets. This component works the following ways to creates indicators:

• Creates a summary for all variables, then establish which of the accessions aim with the parameters that users are needing. The summary option can be: average, median, quartiles (1, 2, 3), percentiles (5, 95) and standard deviation.

Description

As an Advanced user in the web module, I want to be able to search subsets of accessions from the selection of a location on the map and for the system to show the subset of accessions that meet similarities with the characteristics of the selected or entered area and that these are not precisely within the selected area.

Acceptance criteria

1. The system must allow to select an area on the map.
2. The system must allow to enter longitude and latitude values.

Stage 1 – Search by selection on the map: The user selects a polygon on the map which is the area where he wants search the accessions subset that by the nature of their characteristics have similarity with the characteristics of the selected area and then the system shows the accessions list that meet with the filters.

Stage 2 – Search by entering longitude and latitude values: The user enters a longitude and latitude values which are the location where he wants search the accessions subset that by the nature of their characteristics have similarity with the characteristics of the location entered and then the system shows the accessions list that meet with the filters.

Generator extracted values indicator is the module which calculates the indicator for databases in which is just required extract a value from data source, then set the value directly for pixels. The data, which is generated by this module, is saved into the database.

Description

As a user in the web module, I want to be able search subsets of Genebank accessions from of filters typed with a basic semantic in natural language related with the indicators previously loaded, to show a list of accessions that meet these criteria and they are part of one subset.

Acceptance criteria

1. The filters must be related with the indicators previously loaded.
2. Once the user makes the query, the system will activate the option described in the HUS016 related with the download of outcomes.
3. Once the user makes the query, the system will activate the option described in the HUS017 related with the visualization of stats.
4. Once the user makes the query, the system will activate the option described in the HUS018 related with the custom URL of the accession subset created.
5. Once the user makes the query, the system will activate the option described in the HUS019 related with the visualization of map with the accessions location information.
6. Once the user makes the query, the system will activate the option described in the HUS020 related with the filters to accessions from a selection of area of locations in the map.
7. The advanced semantic include: stats data, differences between accessions, values, special qualities of an accession in a certain place, characteristics of an accession.

Stage 1 - Search subset: The basic user must type a query that has at least one taxonomy and one keyword related with the indicators (Passport, environmental, abiotic, biotic, quality, genetic), latterly the basic user must to activates the search option, then the system will query and will shows the outcomes in one list.

Description

As an advanced user in the web module, I want to be able search subsets of Genebank accessions from of filters selecting controls related with the indicators previously loaded, to show a list of accessions that meet these criteria and they are part of one subset.

Acceptance criteria

1. The filters must be related with the indicators previously loaded.
2. Once the user makes the query, the system will activate the option described in the HUS016 related with the download of outcomes.
3. Once the user makes the query, the system will activate the option described in the HUS017 related with the visualization of stats.
4. Once the user makes the query, the system will activate the option described in the HUS018 related with the custom URL of the accession subset created.
5. Once the user makes the query, the system will activate the option described in the HUS019 related with the visualization of map with the accessions location information.
6. Once the user makes the query, the system will activate the option described in the HUS020 related with the filters to accessions from a selection of area of locations in the map.
7. The advanced semantic include: stats data, differences between accessions, values, special qualities of an accession in a certain place, characteristics of a accession.

Stage 1 - Search subset: The user must type a query that has at least one taxonomy and one keyword related with the indicators (Passport, environmental, abiotic, biotic, quality, genetic) and the selection of one control special to advanced search, latterly the user must to activates the search option, then the system will query and will shows the outcomes in one list.

The project requires the following docker images:

• Indicator Database: 4 directories to shared between guess and host: data, logs, configuration, backup. SO: Linux Debian, Version: Mongo
• R container: The container should have 4 directories to shared between guess and host: scripts, inputs, outputs and logs. SO: Linux Debian. Version R: 3. List of package: pacman, raster, rgdal, rgeos, stringr, tidyverse, lubridate, foreach, doSNOW, doParallel, GSIF
• Python Web:
• Subsets API:

The container should have

• 4 directories to shared between guess and host: data, logs, configuration, backup

• SO: Linux Debian

• Version: Mongo

Generator generic indicators is a module which should calculates indicators for all world. This module uses Module time series in order to calculate these indicators. It requires of raster base, which allows to identify the id of all pixels.

Description

As an Advanced user in the web module, I want to be able share accessions subsets outputs via URL, which generates a secure link so that the user can access the outputs of created accessions subset.

Acceptance criteria

Stage 1 - Generate URL: Once the advanced user creates a Genebank accessions subset, the system must generate a new custom URL so that the user can share it.

Description

As an user, I want to be able build a set of custom indicators for each accession from a file previously loaded in the tool, with which latterly the users will may to create Genebank accessions subsets from filters based in them.

Acceptance criteria

• Stage 1 – Load custom indicators file: To build a custom indicator the user must have a file with the following fields: id taxonomy or taxonomy name, indicator (which will add the value) and the indicator value, once him filled out it will proceed to select it in the section custom indicators, the user will run the script that build the custom indicators.
• Stage 2 - Successful operation: For each stage, if the operation flow is successful the system shows a message indicating it.
• Stage 3 - Failed operation: For each stage, if the file structure isn't correctly, the system will show an exception with the message: "The file structure isn't correct."
• Stage 4 - Missing value: For each stage, if the file has a column without value, the indicator for this accession won't create.
• Stage 5 - Generate logs file: For each stage the system will generates a log file, which is described in the HUS007.

Generator passport indicator is a module to index the accessions depending of country, biological status, geographic position, altitude andinstitution. It extracts relevant data from accessions to build passports indicators. It set the pixel index for all accession which have coordinates, it allows that the filtering process can be according to the indicators. The accessions are extracted from Genesys through its Web API.

Description

As an Advanced user in the web module, I want to be see represented on a map the location information of the accessions created in a subset, to visualize geographically how are these accessions distributed.

The logs output should have the following format

TYPE | DATE | DESCRIPTION

TYPE: INFO | ERROR
DATE: yyyy-MM-dd HH:mm:ss

Example:

INFO | 2020-12-04 21:46:19 | Getting accessions passport data for crop(s): african yam bean
ERROR | 2020-12-04 21:46:29 | Error in curl::curl_fetch_memory(url, handle = handle): Timeout was reached: [api.genesys-pgr.org] Connection timed out after 10002 milliseconds

We need to develop the master script, which will run all indicators. The requirements are:

• It should be in R
• It should call functions of other scripts
• It should save data into mongo database

Description

As an Advanced user in the web module, I want to be custom my accessions subsets selecting an area on the map and that the system filters the subset only with the accessions that have match with the location information of selected area.

Acceptance criteria

1. The system must have a rectangle with the world map where the user can draw the preference area.
2. The system must have a section where the user can filter according country, state or county

Exceptions
3. The user must select a valid area.

Description

As an user in the web module, I want to be able select preferred language, to that the tool adapts to the selected language and can offer a better user experience regarding its features.

Acceptance criteria

1. The languages available should be: English and Spanish.

Before to import accessions into the database, we should validate which ones have good coordinates or bad coordinates, then we should add a new flag for those accessions into the database.

• Compare with limits of latitude and longitude
• Compare with shapefile or raster if the point are in water or land

The Subsets API is a Web API, which contains a set of methods to create subsets of accessions based on indicators. The indicators can be previously calculated or custom datasets of the users. It is used like an access point to get access to subsets of accessions. This container just offers one method to obtain subsets of accession, this web method exposes receive a json string in which should be explicit parameters to build the subset.

Search engine is component which allows interpret the natural language of users and translate it to parameters which can be understood by the system. It has two models of natural language processing: the first one is to recognize the intent of users, the second one is to get key words from the sentences, the last one is called named entity recognition (NER). Intent recognition model considers all utterance, while the NER model extracts parameters to build subsets of accessions.

Description
A user wants to be able to import a set of passport indicators for each accession, with which the users will be able later to create Genebank accessions subsets from filters based on these passport indicators.

Acceptance criteria

1. The file with the data extracted of Genesys must be in JSON Format.

• Stage 1 - – Import Genesys passport data:  To import the passport data since genesys to the tool, the administrator user must activate the option “Import accessions passport data since Genesys” and the system will fetch the passport indicators for each accession such as (Location information, tag if the accession is core, institution information biological status). 
• Stage 2 -- Successful operation: For each stage, if the operation flow is successful the system shows a message indicating it.
• Stage 3 -- Failed operation: For each stage, if the file structure isn't correctly, the system will show an exception with the message: "The file structure isn't correct.".
• Stage 4 -- Generate logs file:  For each stage the system will generates a log file, which is described in the HUS007.

Fix the names of folders and other libraries according to new architecture

Description

As an Advanced user in the web module, I want to be able download the outcomes of the subsets of accessions as well as the parameter values that allowed find this set of accessions in different formats, to make a later analysis of the local way with the data obtained.

Acceptance criteria

1. The formats available should be: CSV and Json.
2. The data will be available are: Genebank identification number and DOIs and values indicator.
3. The outcomes stats available to download is: Tables

Stage 1: Once the outcomes of the subsets are shown in list by the search engine, the download option will be visible in the web module and the user will be able to select the preferred download format.

This task comes from #43

Module time series is the component which calculates indicators for all databases which are composed by time series data. It splits a dataset for years, then creates groups by months, with these data it creates a statistical distribution with data for each month, then it shall establish which pixels aim with indicator and set the indicator value for all pixels. Further it creates a summary for all years and months, to do this, it calculates: average, median, quartiles (1, 2, 3), percentiles (5, 95) and standard deviation.

Description
As an administrator user, I want to be able build a set of biotic indicators for each accession, with which latterly the users will may to create Genebank accessions subsets from filters based in them.

Acceptance criteria

• 1. The file generated must be in Shapefile format.

• Stage 1 - Number of water stress day: To create the number of water stress day indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, ndws, collection date),  once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators number of water stress day and latterly him will run the script that build them.

• Stage 2 - Number of days with waterlogging: To create the number of days with waterlogging indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, ndwl, collection date),  once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators number of days with waterlogging and latterly him will run the script that build them.     

• Stage 3 - Extreme daily precipitation: To create the extreme daily precipitation indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, p95, collection date),  once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators extreme daily precipitation and latterly him will run the script that build them.            

• Stage 4 - Average precipitation: To create the average precipitation indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, ravg, collection date),  once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators average precipitation and latterly him will run the script that build them.          

• Stage 5 - Number of days high daytime temperature: To create the number of days high daytime temperature indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, nth, collection date),  once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators number of days high daytime temperature and latterly him will run the script that build  them         

• Stage 6 - Number of days with low minimum temperatures: To create the number of days with low minimum temperature indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, ntl, collection date), once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators number of days with low minimum temperature and latterly him will run the script that build them.            

• Stage 7 - Average minimum temperature: To create the average minimum temperature indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, tn, collection date), once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators average minimum temperature and latterly him will run the script that build them.           

• Stage 8 - Average maximum temperature: To create the average maximum temperature indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, tx, collection date), once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators average maximum temperature and latterly him will run the script that build them.          

• Stage 9 - Average solar radiation: To create the average solar radiation indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, sr, collection date), once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators average solar radiation and latterly him will run the script that build them.              

• Stage 10 - Monthly mean daylength: To create the monthly mean daylength indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, dl, collection date), once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators monthly mean daylength and latterly him will run the script that build them.                                                                                                                                                                           

• Stage 11 - Number of days with high VPD: To create the number of days with high VPD indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, nvpd4, collection date), once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators number of days with high VPD and latterly him will run the script that build them.       

• Stage 12 - Average VPD:  To create the average VPD indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, vpd, collection date),  once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators average VPD and latterly him will run the script that build them.

• Stage 13 - Consecutive dry days: To create the consecutive dry days indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, cdd, collection date), once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators consecutive dry days and latterly him will run the script that build them.  

• Stage 14 - Soil texture type: To create the soil texture type indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, texmht, collection date), once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators soil texture type and latterly him will run the script that build them.     

• Stage 15 - Bulk density: To create the bulk density indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, bldfie, collection date),  once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators bulk density and latterly him will run the script that build them.

• Stage 16 - Cation exchange capacity: To create the cation exchange capacity indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, cecsol, collection date),  once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators cation exchange capacity and latterly him will run the script that build them.

• Stage 17 - Organic carbon content: To create the organic carbon content indicator for each accession the administrator user must have a file with the following fields (cellid, lon, lat, orcdrc, collection date),  once him had filled out it, the administrator user will proceed to select it in the section abiotic indicators organic carbon content capacity and latterly him will run the script that build them.

• Stage 18 - Climate data: To create the climate data indicator for each accession the administrator user must have a file with  the following fields (Id_accesion, minimum temperature, maximum temperature, solar radiation),  once him had filled out it, the administrator user will proceed to select it in the section environmental indicators climate data and latterly him will run the script that build them.

• Stage 19 - Successful operation: For each stage, if the operation flow is successful the system shows a message indicating it.       

• Stage 20 - Failed operation: For each stage, if the file structure isn't correctly, the system will show an exception with the message: "The file structure isn't correct."    

• Stage 21 - Missing value: For each stage, if the file has a column without value, the indicator for this accession won't create.  

• Stage 22 - Generate logs file:  For each stage the system will generates a log file, which is described in the HUS007.

• Processing passport data (@juangarc )

Description

As an administrator user, I want to be able generate a logs file for each executed process, in order to have a record of the events and actions that occurred to the process during its execution.

Acceptance criteria

1. The file must contain the following data: process date, method, comment and state (processing, finished, error, stopped).                                                                                                              

Stage 1 – Generate logs file: For each action that is executed for the user in the generator module, the system has to create a logs file and that it will can be visualized for the user.

Description

As an advanced user in the web module, I want to be able see in charts the stats that has been generated about the indicators of each accession, to visualize and analyze the behavior these according to their use.

Acceptance criteria

1. The chart available to stats with time series should be: Line plot.
2. The chart available to categorical stats should be: Bar plot.
3. The chart available to quantitative stats should be: Tables.
4. The statistics available should be: median, arithmetic average, quartiles, percentiles (5, 95) and standard deviation.

Indicator DB is a database which stores generic, specific per crop and extracted values indicators. It also contains passport data of accession available in Genesys. It is the source to create subsets of accessions.

The raster base we are working with does not cover all the coordinates available in the accessions data. This results in NA cellid even for non NA coordinates.

Description

As an administrator user, I want to be able build a set of quality indicators for each accession, with which latterly the users will may to create Genebank accessions subsets from filters based in them.

Acceptance criteria

Stage 1 - Type of consumption: To create the type of consumption indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators type of consumption and latterly him will run the script that build them.                             

Stage 2 - Nutritional quality:  To create the nutritional quality indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators nutritional quality and latterly him will run the script that build them.
                                                                                                                                           
Stage 3 - Use:  To create the use indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators use and latterly him will run the script that build them.                                                                                                                                                                   

Stage 4 - Type of material:  To create the type of material indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators type of material and latterly him will run the script that build them.                                                                                              

Stage 5 - Use value by type of livestock: To create the use value by type of livestock indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators use value by type of livestock and latterly him will run the script that build them.
                                                                                                                                                                                 
Stage 6 - Soil restoration: To create the soil restoration indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators soil restoration and latterly him will run the script that build them.                                                                                                                                                         

Stage 7 - Starch content: To create the starch content indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators starch content and latterly him will run the script that build them.
                                                                                                                                                         
Stage 8 - Dry matter content: To create the dry matter content indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators dry matter content and latterly him will run the script that build them.
                                                                                                                                          
Stage 9 - Taste:  To create the taste indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators taste and latterly him will run the script that build them.                                                                                                                                                                                                                                                          
Stage 10 - Zinc concentration: To create the zinc concentration indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators zinc concentration and latterly him will run the script that build them.                               

Stage 11 - Wheat end-use quality: To create the wheat end-use quality indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators wheat end-use quality and latterly him will run the script that build them.
                                           
Stage 12 - Vitamin A concentration: To create the vitamin A concentration indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators vitamin A concentration and latterly him will run the script that build them.
                                                                                                  
Stage 13 - Popcorn quality: To create the popcorn quality indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators popcorn quality and latterly him will run the script that build them.                                                                                                                                                                                                                                                          
Stage 14 - Semolina pigment concentration: To create the semolina pigment concentration indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators semolina pigment concentration and latterly him will run the script that build them.                                                                                                                                                                                    
Stage 15 - Grain protein concentration: To create the grain protein concentration indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators grain protein concentration and latterly him will run the script that build them.
                                                                                                                                                                                      
Stage 16 - Chipping or grying quality: To create the chipping quality indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators chipping quality and latterly him will run the script that build them.
                                                                                                                                
Stage 17 - Flesh or skin color:  To create the flesh color indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators flesh color and latterly him will run the script that build them.               

Stage 18 - Nutrient properties:  To create the nutrient properties indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators nutrient properties and latterly him will run the script that build them.

Stage 19 - Digestibility:  To create the digestibility indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators digestibility and latterly him will run the script that build them.
                                                                                                                                                                            
Stage 20 - Amylose content: To create the amylose content indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators amylose content and latterly him will run the script that build them.
                                                                                                                              
Stage 21 - Grain size:  To create the grain size indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators grain size and latterly him will run the script that build them.                                                                                                                          

Stage 22 - Nutritious: To create the nutritious indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators nutritious and latterly him will run the script that build them.
                                                                                                                                                                                               
Stage 23 - Physical characteristics:  To create the physical characteristics indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators physical characteristics and latterly him will run the script that build them.
                                                                                                                      
Stage 24 - Protein quality:  To create the protein quality indicator for each accession the administrator user must have a file,  once him had filled out it, the administrator user will proceed to select it in the section quality indicators protein quality and latterly him will run the script that build them.                                                                                                                                                                                                                                                         
Stage 25 - Protein content: To create the protein content indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators protein content and latterly him will run the script that build them.                         

Stage 26 - Magnesium content:  To create the magnesium content indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators magnesium content and latterly him will run the script that build them.

Stage 27 - Iron content:  To create the iron content indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators iron content and latterly him will run the script that build them.
                                                                                                                                                                                                
Stage 28 - Seed size: To create the seed size indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators seed size and latterly him will run the script that build them. 
                                                                                                                                                                   
Stage 29 - Macronutrient content:  To create the macronutrient content indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators macronutrient content and latterly him will run the script that build them.
                                                                                                                                                    
Stage 30 - Low ODAP: To create the low ODAP indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators low ODAP and latterly him will run the script that build them.                                                                                                                                                                                                                                                         
Stage 31 - Grain nutrients:  To create the low ODAP indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators low ODAP and latterly him will run the script that build them.                                                                                                                                                                                                                                                         
Stage 32 - Forage quality:  To create the forage quality indicator for each accession the administrator user must have a file, once him had filled out it, the administrator user will proceed to select it in the section quality indicators forage quality and latterly him will run the script that build them.                                                                                                                                                               
Stage 33 - Soil data: To create the soil data indicator for each accession the administrator user must have a file with the following fields (Id_accesion, soil type, texture type, salinity in the soil, fertility indicators),  once him had filled out it, the administrator user will proceed to select it in the section environmental indicators soil data and latterly him will run the script that build them.                                                                                                                                                                                                                                                                
Stage 34 - Successful operation: For each stage, if the operation flow is successful the system shows a message indicating it.                                                                                                                                                          

Stage 35 - Failed operation: For each stage, if the file structure isn't correctly, the system will show an exception with the message: "The file structure isn't correct."                                                             

Stage 36 - Missing value: For each stage, if the file has a column without value, the indicator for this accession won't create.                                                                                                                                                                                                                                                                                    
Stage 37 - Generate logs file: For each stage the system will generates a log file, which is described in the HUS007.

• @khadijaAouzal will improve the explanation about extracted values
• @bmora-0110 should add the oversea level indicator in the table of extracted values indicators

Description

As an user in the web module, I want to be able select preferred language, to that the tool adapts to the selected language and can offer a better user experience regarding its features.

Acceptance criteria

1. The languages available should be: English and Spanish

Master script is responsible for executing the processes that build indicators. This orchestrates the construction of indicators according to the given request, here users must specify the type of indicators you want to build, then the script will search the Genesys API for the main data of the accessions (passport data) and then send them to the construction of the indicator through the Parallelization Component according to the type of indicator set previously identified, once the indicator has done the process, it returns the calculated indicator of the accession. This component saves data of indicators into the indicator database.

The container should have 4 directories to shared between guess and host: scripts, inputs, outputs and logs

SO: Linux Debian

Version:

List of package:
pacman, raster, rgdal, rgeos, stringr, tidyverse, lubridate, foreach, doSNOW, doParallel, GSIF

@bmora-0110

• Extract climate data for the new raster base. It is when we need to calculate some climate indicators for pixels which weren't include in CHIRPS, we have the capacity to get this new data

@khadijaAouzal

• She will develop a new module inside of web api which allow to analyze data with multivariable approach

@juangarc

• He will implement the function of @khadijaAouzal will develop

Description

As an administrator user, I want to be able build a set of genetic indicators for each accession, with which latterly the users will may to create Genebank accessions subsets from filters based in them.

Acceptance criteria

Stage 1 - Discrete variables and continuous one: To build a discrete variable and continuous the user must have a file with the following fields (V1, V2, V3, V4, V5, V6, cluster, ig, country, lon, lat), once him filled out it will proceed to select it in the section genetic indicators discrete variables and continuous latterly he will run the script that build the genetic indicators.

Stage 2 - Successful operation: For each stage, if the operation flow is successful the system shows a message indicating it.

Stage 3 - Failed operation: For each stage, if the file structure isn't correctly, the system will show an exception with the message: "The file structure isn't correct."

Stage 4 - Missing value: For each stage, if the file has a column without value, the indicator for this accession won't create.

Stage 5 - Generate logs file: For each stage the system will generates a log file, which is described in the HUS007.

Implement a module frontend friendly with the users for that their can use the features of the applications.

Description

As an user in the web module, I want to be able sign in into website with same credentials of Genesys website, to be recognized by the tool, to be able to store the accessions subsets generated and to load own information of the indicators in the session.

Acceptance criteria

1. The username and password must be the same from login to Genesys.
2. The system must store in a cookie the same Genesys login token.

Stage 1: Once the user to type the same username and password of Genesys login, when the user activates the login option, then it will load the web module with user basic information.

Stage 2: Once the user to enter the credentials for login, when the user activates the login option if the username or password is incorrect, then the system will show the exception: "Username or password incorrect".

Description

As an Advanced user in the web module, I want to be able see stats of the subsets created, to analyze and visualize the outcomes of generated subsets outcomes since different perspectives.

Acceptance criteria

1. The plots must be displayed are: median, average, quartiles, percentiles, range variance, cumulative absolute frequency, relative frequency and cumulative relative frequency.                             
2. The chart available to these stats are: Bar chart, horizontal bar chart, grouped bar chart.

Generator specific per crops indicator is a module which calculates indicators for specific crops. It requires a special parametrization for each crop, further requires to establish which is the grown up areas for those crops. It uses Module time series in order to calculate these indicators.

We have a doubt about what we should do with accessions without coordinates from genesys

1. We just ignore them into our filters
2. We should georeference them and work with the new coordinates

Subsets DB contains data about user sessions, such as preferences, language, and other settings. It also saves information with custom URLs that refer to subsets of accessions and that users can share with others.