Giter Club home page Giter Club logo

abalone's Introduction

Abalone Analytics

The Bodega Marine Laboratory's White Abalone captive breeding program is working to prevent the extinction of the White Abalone (Haliotis sorenseni), an endangered marine snail. White abalone are one of seven species found in California and are culturally significant to the native people of the area. White abalone were perilously overfished throughout the 20th century, resulting in a 99 percent population decrease by the end of the 1970s. This group is working to reverse their decline and have already seen some great success, they currently have more abalone in the lab than exist in the wild!

Ruby for Good is supporting these efforts by developing a data tracking and analytics system for Abalone population trends, mortality rates, and breeding programs to help save this species from extinction.

Getting Started

Prerequisites

This application is built on following and you must have these installed before you begin:

  • Ruby (2.6.3)
  • Rails (5.2)
  • PostgreSQL (tested on 9.x)

Setup

After forking this repo and cloning your own copy onto your local machine, execute the following commands in your CLI:

gem install bundler
bundle install
rake db:create
rake db:migrate
rake db:seed

Then, run bundle exec rails s and browse to http://localhost:3000/.

Running Background Jobs

The app uses the gem delayed_job for processing CSVs. To run background jobs, run the following command in your CLI:

rake jobs:work

To confirm background jobs are processing, try uploading a CSV at http://localhost:3000/file_uploads/new. You should see the job complete in your CLI and see the file upload results here at http://localhost:3000/file_uploads.

To see detailed logs from background jobs, run:

tail -f log/delayed_job.log

Contribute

We would love to have you contribute! Checkout the Issues tab and make sure you understand the acceptance criteria before starting one. Before you start, get familiar with important terms, how the app works right now, sample data and the steps to MVP below:

Get Familiar with the App

This app is still in early stages of development (MVP). We have defined our MVP and additional milestones here

Take a look at the current Issues, which lay out our path to MVP. Feel free to assign one to yourself and take it on! If you have any questions about requirements, post your question in the issue or email Ellen Cornelius at [email protected].

Development

We have included the Annotate gem in this project, for better development experience. It annotates (table attributes) models, model specs, and factories.

The annotate task will run automatically when running migrations. Please see lib/tasks/auto_annotate_models.rake for configuration details.

If it does not run automatically, you can run it manually, on the project root dir, with:

annotate

Check out their Github page for more running options.

The Problem

Our stakeholder, the Bodega Marine Laboratory, has more data that they can keep track of! They want to have a central data repository for all of their abalone captive breeding data instead of just spreadhseets. It is hard to run reports and anlytics on the data when it's not all in one place.

The Solution

We are building an app which has the following capabilities:

  1. Store Raw Data: There are several different types of CSVs that the lab has been amassing (Mortality Tracking Data, Pedigree Data, Population Estimate Data, Spawning Success Data, Tagged Animal Assessment Data, Untagged Animal Assessment Data, and Wild Collection Data). Examples of these CSVs can be found in the db/sample_data_files directory.
  2. Import CSVs: Users are able to import single and bulk CSVs. Users should generally submit cleaned CSVs, but the app should alert users if there are parsing problems and which row(s) need to be fixed.
  3. Display Charts and Analytics: For MVP, we would like to display a Histogram binned in 1cm increments of different body lengths for a certain cohort or group of cohorts.
  4. Export CSVs: TBD.

Jargon

  • Tag number(s), date = e.g. Green_389 from 3/4/08 to 4/6/15 We sometimes tag individuals; however, not all individuals have tags. We can't tag individuals until they are older than one year old because they are too small. Generally a color, a 3-digit number and dates that tag was on. Sometimes tags fall off. It can be logistically challenging to give them the same tag, so they sometimes get assigned new tags. Also, occasionally tags have another form besides color_### (e.g., they have 2 or 4 digits and/or have no color associated with them), and sometimes they are something crazy like, "no tag" or "no tag red zip tie" for animals that lived long ago ... though I suppose we could re-code those into something more tractable.
  • Individual ID = YYYY_MM_DD_color_### The individuals' ID is ithe date it was spawned followed by its initial tag color and 3-digit number
  • Shellfish Health Lab Case Number = SF##-## Animals from each spawning date and from each wild collection have a unique case number created by California's state Shellfish Health Laboratory (SHL). Sometimes animals from a single spawning date have more than one SHL number.
  • Cohort = place_YYYY This is how the lab coloquilly refers to each of their populations spawned on a certain date. It's bascially a note/nickname for each group of animals with a particular SHL #/spawning date.
  • Institution = e.g. BML from 6/5/13 - 11/20/14 Animals move around among a finite number of partner institutions (it is possible for new facilities to be added, but it only happens about once every few years).
  • Holding area = e.g. Juvenile Rack 1 Column A Trough 3 from 3/4/15 - 6/2/16 This is the tank space by date. This is a note. The types of input will vary significantly within a facility and over time.

See a full data dictionary here.

CSV Upload Architecture

file upload architecture diagram

Deployment

The application is currently deployed on a DigitalOcean droplet via Capistrano. Once your public SSH key has been added to the appropriate user on the necessary server(s), use bundle exec cap production deploy to deploy the application, run migrations, and restart the Puma application server. Puma is reverse-proxied behind Nginx. The Nginx configuration is currently maintained outside of the Rails development pipeline. Currently live at abalone.blrice.net.

And Don't Forget...

...that Gary needs you.

a white abalone

Photo credit: John Burgess/The Press Democrat

abalone's People

Contributors

bgarr avatar corneliusellen avatar fjbotto avatar glassjoseph avatar jcavena avatar jgysland avatar jtu0 avatar marc avatar openmailbox avatar rafaltrojanowski avatar seanmarcia avatar thunderheavyindustries avatar yarmiganosca avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.