njcoast / cyberspatial Goto Github PK
View Code? Open in Web Editor NEWNJCoast CyberSpatial Framework based on GeoNode
NJCoast CyberSpatial Framework based on GeoNode
Disable Data and Maps dropdowns, with the requirement that when the user is logged in he/she sees both dropdowns, otherwise only About is visible (check bootstrap if it supports that).
Configuration of Geoserver endpoint to accept vector tiles. There is a vector tiles tutorial in the geoserver documentation.
We need to have an understanding how the GIS components scale with size.
Configure Kubernetes secrets to correctly deploy Django Keys and other sensitive data.
Scripts to automatically generate documentation from embedded code comments using Sphinx.
The tile workflow need to be extensible to generate 3D tiles using one of the emerging standards at the same time we generate the contours. This could potentially be used with 3D terrain mapping in future version of the model. One emerging specification is the Cesium Tile specification that has been submitted to a OGC working group.
Complete documentation with focus on system architecture and how scaling can be accomplished. Use diagram in the template.
https://docs.google.com/document/d/1KeVk4gw1cDi9TqTS5528Ty0bC0C2cekiQKByuRf-vc8/edit
Configure 3 Ubuntu VM's with one as public facing master and two slaves.
These will be used to have the website online and also as general development setup.
hostnames are:
specs:
Ubuntu 16.04 LTS with 2 cores, 2 GB RAM and 25 GB disk
Only two rules have been added: one allowing SSH from anywhere (still restricted in /etc/hosts.allow though) and another allowing access from our backup server for the nightly TiBS tape backups.
Login with ND netID and passwords.
Chris and James have sudo.
Django models need to be added to GeoNode to support Role based permissions in data access. These role data models would also support search (discovery) of datasets based upon rules associated with each role that specifies data type preferences.
Replicating Maploom functionality to GeoNode, look at Leaflet Draw: https://leaflet.github.io/Leaflet.draw/docs/leaflet-draw-latest.html
Kristina provided the CSS for the user dashboard. CSS and custom graphic files attached.
Mockup:
https://app.zenhub.com/files/88267239/b00ca30b-43aa-43ad-828d-ffbe88bf22c5/download
Refactoring python model code to generate GeoJSON vector tiles from matplotlib contours. There is a utility code that can generate contours called geojsoncontour.
Relevant links are:
Change to generating vector tile sets:
Mapbox tiles
Generate GeoJSON contours from matplotlib,
Matplotlib Library
[Matplotlib basemap utility)[https://github.com/matplotlib/basemap]
Geoserver python configuration library.
Mapbox geojson vector tile server
Youtube talk that explains vector tile implementation in GeoServer.
Stable cyberinfrastructure fully containerized and ready for population.
Architecture:
Software stack:
Database footprint:
being used for API and user documentation.
Configuration of persistent disk storage snapshots for Google Cloud infrastructure. Insure that instances of GeoNode and Geoserver can be live migrated.
As a user
I want to use the map with various tools to manipulate, share and store map/layer data.
We are not concerned about roles for now, just note that there are 3 different roles for which we need to adopt a permission schema to control features. Right now, we just want to have a user being able to use the site.
Mockup: http://www.crc.nd.edu/~kfurse/nj-coast_front-mock/map-p.html
AC:
Configure Kubernetes to deploy the Docker-SNORT image configuration. This will also necessitate including the proper ssl certificates for njcoast.us.
Celery tasks to trigger ElasticSearch indexes of GeoNode metadata don't appear to be configured correctly. ElasticSearch container need to be added to Kubernetes deployment scripts.
As a PO
I want to have a realistic view of all tasks
so I can schedule additional resources, track the budget and over time task progression.
Tim and Swapna are conducting following scans:
11/21 nmap
11/28 ZAP (Admin account needed)
12/4 Code review
? Snort
Extend secrets persistent volume to include Django Environment variables and the secret deployment key.
Geoserver allows persistent base layers to be stored in the local_settings.py and automatically loaded into new instances. ArcGIS layers can also be added in this way. See the Geonode mailing list for a discussion. A sample base layer is:
Add ESR world ocean reference base layer to geoexplorer
source: new ol.source.XYZ({
url: 'http://server.arcgisonline.com/ArcGIS/rest/services/' +
'Ocean/World_Ocean_Reference/MapServer/tile/{z}/{y}/{x}'
})
What to change in production:
http://docs.geonode.org/en/dev/deploy/production.html
How to add XYZ:
GeoNode/geonode#1286
{
"source": {"ptype": "gxp_olsource"},
"type":"OpenLayers.Layer.XYZ",
"args":[
"hot", ["http://a.tile.openstreetmap.fr/hot/${z}/${x}/${y}.png", "http://b.tile.openstreetmap.fr/hot/${z}/${x}/${y}.png", "http://c.tile.openstreetmap.fr/hot/${z}/${x}/${y}.png"], {"transitionEffect": "resize","attribution": "osm_attribution"}],
"visibility": True,
"fixed": True,
"group":"background"
},
Kristina provided the CSS for the landing page. CSS and custom graphic files attached.
http://www.crc.nd.edu/~kfurse/nj-coast_front-mock/about.html
https://app.zenhub.com/files/88267239/d28fd98b-ac83-4a0f-a662-5b4acb455e64/downloadhttps://app.zenhub.com/files/88267239/5feb9a05-7215-4b40-ac03-1e5e470d87b8/download
Comment out People and Groups in the About dropdown and have the text as single page.
Notes from Kristina:
Minicube fails to mount the host directories specified in the Kubernetes configuration files for minikube. This prevents both postgresql and geoserver persistent data volumes from have the correct data structure when the pods start. It is unknown if this issue exists with the GKE persistent volumes.
It may be possible to use Kubernetes for primary job management of computational models. We don't want extra resources that are not needed, but to scale as necessary.
Migrate Docker Gateway image for nginx configuration in kubenetes deployment.
NGINX/Gunicorn/Django is not correctly routing the Geoserver endpoint to njcoast.us/geoserver and njcoast.us/geoserver/web.
Remove links for all 3 planer buttons on the frontpage on dev server.
As a dev
I want to set up kubernetes in AWS
so that I can bring njcoast.us online.
Check out AWS with Kubernetes to replace GKE.
A decision needs to be made for whether we go with the ND account or create our own.
Setup environment for Django development. Please talk to James about the required stack and existing dependencies.
Also he needs to show where to commit your code (repo/branch).
As a user
I want to go through a tutorial/training
so that I get a better understanding of the tools and different features.
AC
1.How will the tutorial be triggered/ accessed from?
2.What is the scope of the tutorial?
2.1.Account creation? explaining the 3 different roles
2.2.Maps/Layers/Annotation tools/ Share/Create a Map,Layer/Download
2.3.Storm Hazard Projection Tool
Auto-deployment testing scripts for TravisCI or for Jenkins CI test servers.
Check why the map preview icon for the test map is not being displayed.
http://njcoast.us/maps/?limit=100&offset=0
Create a new project plan for the SLA in Ominplan with an updated timeline of all Milestones M4 to M8.
We need to setup a build server. Investigate Travis
https://blog.travis-ci.com/
http://thylong.com/ci/2016/deploying-from-travis-to-gce/
Some shape files are quite large. Receiving a "413 Request Entity too large" from NGINX when uploading a shape file via the layer upload system. This can be configured through the nginx.conf file used to build the container.
https://www.cyberciti.biz/faq/linux-unix-bsd-nginx-413-request-entity-too-large/
Kristina provided the CSS for the landing page. CSS and custom graphic files attached.
Mockup: http://www.crc.nd.edu/~kfurse/nj-coast_front-mock/
https://app.zenhub.com/files/88267239/aa79619a-aeb5-4164-9b99-67537d5872ad/download
https://app.zenhub.com/files/88267239/23132582-9fc2-4093-a301-31e472c9a88a/download
We want a dummy template in the repo but actual deployment information secure in the CRC redmine instance.
Explore roles EM, Planner and First responders what layers they can access.
Duplicate: https://app.zenhub.com/workspace/o/njcoast/cyberspatial/issues/123
Setup endpoint for authentication for T&M Map layers.
Design of Restful components for bot the Job management and individual computational models. We need to consider the extra metadata necessary to describe the model result's response in known controlled vocabularies. The Earthcube xdomes.org is probably a good starting point for building out the controlled vocabularies.
Addition of map editing and annotation features to GeoNode instances. Basis for code could be Maploom or the Mapstory project that has a GitHub Organization.
We need to fill out the following doc:
https://drive.google.com/drive/u/0/folders/0B9fGRYX4RNaSTE1XREpSZU1QS2s?ths=true
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.