nasa-petal / petal Goto Github PK
View Code? Open in Web Editor NEWArtificial Intelligence and Natural System Design Tool
Home Page: https://nasa-petal.github.io/PeTaL/
License: The Unlicense
Artificial Intelligence and Natural System Design Tool
Home Page: https://nasa-petal.github.io/PeTaL/
License: The Unlicense
this is a test
Containerize PeTaL into a single startup script, i.e. using docker-compose to spin up the neo4j server and potentially also to pre-fill the database before starting the Django site.
We need more data in Petal
Right now all the cards are on the left side so on the right side there's more space. Centering all the elements will look more good. And pagination on the bottom is not working properly
Enhancement
I will center the cards and the search bar. Font size are small and all seems very close, so I will increase padding too.
Pagination on the bottom is not working properly, so I will fix to just show if there's another page else they will be disabled
I will use Material-UI as this project is already using it and will focus on making it more user-friendly and aesthetic
Right now it's look like this...
CSV write Bug in OptimizedCatalog.py
Current
"Craseonycteris thonglongyai (Hill,1974)" -> Craseonycteris thonglongyai (Hill | 1974)
Should be
"Craseonycteris thonglongyai (Hill,1974)" -> Craseonycteris thonglongyai (Hill,1974)
When OptimizedCatalog is run, it creates csv files from json objects to load into neo4j. These csv files were inaccurate due to commas within the JSON values. The new code should work with commas.
All
Operating System: Windows 10
Python environment: Anaconda Python 3.7.1
Packages:
arrow==0.15.7
asgiref==3.2.10
beautifulsoup4==4.9.1
bibtexparser==1.2.0
bitflow==0.6.0
blis==0.4.1
boto==2.49.0
boto3==1.14.10
botocore==1.17.10
catalogue==1.0.0
certifi==2020.6.20
chardet==3.0.4
click==7.1.2
cycler==0.10.0
cymem==2.0.3
Cython==0.29.14
Django==3.0.7
docutils==0.15.2
efficientnet-pytorch==0.6.3
en-core-web-sm==2.3.1
fake-useragent==0.1.11
future==0.18.2
gensim==3.8.3
idna==2.9
importlib-metadata==1.7.0
jmespath==0.10.0
joblib==0.15.1
kiwisolver==1.2.0
lxml==4.5.1
matplotlib==3.2.2
murmurhash==1.0.2
neo4j==4.0.0
neobolt==1.7.17
nltk==3.5
numpy==1.19.0
pandas==1.0.5
Pillow==7.1.2
plac==1.1.3
plotly==4.8.1
preshed==3.0.2
pyparsing==2.4.7
PySocks==1.7.1
python-dateutil==2.8.1
pytz==2020.1
regex==2020.6.8
requests==2.24.0
retrying==1.3.3
s3transfer==0.3.3
scholarly==0.3.3
scipy==1.4.0
seaborn==0.10.1
six==1.15.0
smart-open==2.0.0
soupsieve==1.9.6
spacy==2.3.2
sqlparse==0.3.1
srsly==1.0.2
stem==1.8.0
thinc==7.4.1
torch==1.5.1+cpu
torchvision==0.6.1+cpu
tqdm==4.46.1
urllib3==1.25.9
wasabi==0.7.1
wikipedia==1.4.0
zipp==3.1.0
Find papers with data we want to add to PeTal
Extract results from said papers
Categorize subject of study into engineering terms (seal whiskers might be under fluid dynamics, capillaries might be under heat transfer)
Have certain 'stats' recorded based on engineering term of interest (for biomimetic studies on capillaries or other fluid flow structures, stats might include Reynolds number, Nusselt, or Grashof)
Remove duplicate/unneeded static files and organize them into one location
Duplicate/unneeded .jpg, .png, .js, .json files.
Need to organize into one location, or have a set template for them.
Vik and PAht has the files
Hey Lucas,
Hate to ask you another question, but am I possibly missing some code or data here? I ran
./run config/mock_species_articles.json
and then after checking that the database actually had data in it. I ran
./run config/search.json
to create the index file. I got this error message
AttributeError: 'Result' object has no attribute 'records'
Documentation about the error message is attached! I really appreciate your time if you could help me with this!
AttributeError 'Result'object has no attribute 'records'.pdf
Based on the fields that are to be populated, a content extractor (perhaps from AllenNLP) would be used to generate a score of whether the inputted answer is correct with regards to the text and the previous answers that are in the system.
As an Engineer, I want to use a guided search (dropdowns) of functions to query for articles, species, etc.
Basically transfer legacy version bird_e2b.html from Flask to Django. Connect to neo4j database.
Based on the fields that are to be populated, a content extractor (perhaps from AllenNLP) would be used to generate a score of whether the inputted answer is correct with regards to the text and the previous answers that are in the system.
Store raw results from neo4j server in central accessible location.
Create optimal UI for combining the UX of all three visualization tools with a single set of data.
Edit the bitflow code to recursively recognize PeTaL modules, take multiple directories as input, and give more helpful error messages when modules aren't found or have errors.
Develop code to mine the following and store into. Store the data into NEO4J
Connect PeTaL UI inputs with NLP and Vision.
PeTal UI should...
Wire-frame - https://projects.invisionapp.com/freehand/document/p5uNMEgWJ
Flow Diagram - [PeTaL Flow-Diagram v3.0].pptx](https://github.com/nasa/PeTaL/files/5108889/PeTaL.Flow-Diagram.v3.0.Autosaved.pptx)
Mockup - PeTaL Mockup v1.2.pptx
PeTaL Mockup v3.0 [Autosaved].pptx
Different files inside module/* have manually (in the python code) entered connection strings. We need to change this so that it uses the same configuration file (json)
Example files:
site/search/bird.py
modules/libraries/download_wikipedia/wiki_to_neo.py
There may be other files that need updating.
Updates sass files and recompile styles.css
Remove unneeded and unused .scss files from static/sass folder. Update paths in the .scss files. Recompile styles.css based on the edited .scss files.
N/A
Create UI to enable user inputs...
Connect to neo4j database
<A brief summary of the issue (used for release notes).>
For bugs, explain what is happening and how it differs from the expected behavior.
For enhancements, include a description of the proposed enhancement.
For bugs, if at all possible, provide a short snippet which reproduces the issue. Or if it is a user interface issue, include the steps needed to cause the bug and include
screenshots as needed.
For enhancements, if possible, give some suggestions for fixes. This could be in the form of code or UI mockups.
<For bugs, list the environment where the bug was noticed. Can say "All" if the bug is not environment specific. For enhancements, use N/A>
Operating System: <e.g. OS X 10.14.6, Windows 10, Ubuntu 16.04>
Python environment: <e.g. Anaconda Python 3.7.1>
Packages: <versions of PeTaL, and other pertinent Python packages. You can just paste the output of pip freeze
or conda list
here.>
Get an error when running this:
./run config/mock_species_articles.json
Anyone else getting this error when starting up the Django web server? I don't see a data/index file in the repo anywhere, am I supposed to generate this file somehow?
https://github.com/nasa/PeTaL/blob/13d280b79343d1858cddef13391c74e88a13025b/site/bird/search.py#L25
Poetry offers modern python package management and wraps both venv and pip. It also makes distribution as a library much simpler.
Guide to writing user personas
User Personas will be documented here:
https://docs.google.com/document/d/1In5t369vErfVZKHZMbjynCx090P0PYUmrEsZtN3tKDM/edit
Add autocomplete capability to the BIRD search tool
The main goal is to create UI that will display an autocomplete list based on a string/list of selections given. Creating the processing of the input (NLP) to decide what terms or set of words to list as selections is not the goal of this enhancement.
Possibly similar to Google's UI
N/A
UI: search form, results output.
Backend: get list of results to return.
A new branch is needed so external contributors may submit pull requests. This is so repository maintainers have a single branch to use for reviewing all external contributions.
Add Maju's segmentatioin work. Vik or Paht can send you the files
BIRD dropdown menu lists will be populated with relevant Neo4j data.
Enhancement
Three dropdown menu's should be created. Functions, Species, Environment.
Here is a link as an example end product for the species dropdowns.
Bugs when parsing and loading COL (Catalogue of Life) data into neo4j
Problems in OptimizedCatalog.py
name:
includes author and date in addition to the scientific name.Most species and taxon nodes are not connected
All
Operating System: Windows 10
Python environment: Anaconda Python 3.8.3
Packages:
arrow==0.15.7
asgiref==3.2.10
beautifulsoup4==4.9.1
bibtexparser==1.2.0
bitflow==0.6.0
boto==2.49.0
boto3==1.14.10
botocore==1.17.10
certifi==2020.6.20
chardet==3.0.4
click==7.1.2
cycler==0.10.0
Cython==0.29.14
Django==3.0.7
docutils==0.15.2
efficientnet-pytorch==0.6.3
fake-useragent==0.1.11
future==0.18.2
gensim==3.8.3
idna==2.9
jmespath==0.10.0
joblib==0.15.1
kiwisolver==1.2.0
lxml==4.5.1
matplotlib==3.2.2
neo4j==4.0.0
neobolt==1.7.17
nltk==3.5
numpy==1.19.0
pandas==1.0.5
Pillow==7.1.2
plotly==4.8.1
pyparsing==2.4.7
PySocks==1.7.1
python-dateutil==2.8.1
pytz==2020.1
regex==2020.6.8
requests==2.24.0
retrying==1.3.3
s3transfer==0.3.3
scholarly==0.3.3
scipy==1.4.0
seaborn==0.10.1
six==1.15.0
smart-open==2.0.0
soupsieve==1.9.6
sqlparse==0.3.1
stem==1.8.0
torch==1.5.1+cpu
torchvision==0.6.1+cpu
tqdm==4.46.1
urllib3==1.25.9
wikipedia==1.4.0
Implement image segmentation using EfficientNet
Implement object detection using EfficientNet
Send results (taxonomy, patterns, objects) to UI
Store results in neo4j
Find the data necessary to cluster and extract data type, shape and size in order to accurately create topic modeler
Pull Kunle's branch of petal https://github.com/kakinpe1u/PeTaL/tree/feature_elasticsearch
and https://github.com/kakinpe1u/PeTaL/tree/feature_indexer
Try to see how we can integrate Kunle's elastic search with Petal.
If we can't integrate then we need to know.
This needs to be done ASAP.
we'll move current master branch to a new branch, and use master branch to check in working, tested code.
Based on the fields that are to be populated, a content extractor (perhaps from AllenNLP) would be used to generate a score of whether the inputted answer is correct with regards to the text and the previous answers that are in the system.
Error when scraping for JEB articles and storing in neo4j
Problems with JEBModule.py.
Only scrapes first article result page. Not the successive pages.
When adding edges to existing nodes, code would not create/add anything into neo4j database.
UUID assignment problem when storing data to neo4j database.
Run a config .json file from main.py with NormalCatalog and JEBModule in the bitflow:whitelist.
All
Operating System: Windows 10
Python environment: Anaconda Python 3.8.3
Packages: all packages in PeTaL poetry package manager
react with material-ui perhaps? if a full fledged app UI framework like material-ui is too heavy for our MVP (and probably is) I'm ok with just using SASS -> CSS.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.