Giter Club home page Giter Club logo

org.geppetto's Introduction

OpenWorm

Docker Image CI Docker Image Test - quick Docker Image Test Build - Intel drivers

About OpenWorm

OpenWorm aims to build the first comprehensive computational model of Caenorhabditis elegans (C. elegans), a microscopic roundworm. With only a thousand cells, it solves basic problems such as feeding, mate-finding and predator avoidance. Despite being extremely well-studied in biology, a deep, principled understanding of the biology of this organism remains elusive.

We are using a bottom-up approach, aimed at observing the worm behaviour emerge from a simulation of data derived from scientific experiments carried out over the past decade. To do so, we are incorporating the data available from the scientific community into software models. We are also forging new collaborations with universities and research institutes to collect data that fill in the gaps.

You can earn a badge with us simply by trying out this package! Click on the image below to get started. OpenWorm Docker Badge

Quickstart

We have put together a Docker container that pulls together the major components of our simulation and runs it on your machine. When you get it all running it does the following:

  1. Run our nervous system model, known as c302, on your computer.
  2. In parallel, run our 3D worm body model, known as Sibernetic, on your computer, using the output of the nervous system model.
  3. Produce graphs from the nervous system and body model that demonstrate its behavior on your computer for you to inspect.
  4. Produce a movie showing the output of the body model.

Example Output

Worm Crawling

NOTE: Running the simulation for the full amount of time would produce content like the above. However, in order to run in a reasonable amount of time, the default run time for the simulation is limited. As such, you will see only a partial output, equivalent to about 5% of run time, compared to the examples above. To extend the run time, use the -d argument as described below.

Installation

Pre-requisites:

  1. You should have at least 60 GB of free space on your machine and at least 2GB of RAM
  2. You should be able to clone git repositories on your machine. Install git, or this GUI may be useful.

To Install:

  1. Install Docker on your system.
  2. If your system does not have enough free space, you can use an external hard disk. On MacOS X, the location for image storage can be specified in the Advanced Tab in Preferences. See this thread in addition for Linux instructions.

Running

  1. Ensure the Docker daemon is running in the background (on MacOS/Windows there should be an icon with the Docker whale logo showing in the menu bar/system tray).
  2. Open a terminal and run: git clone http://github.com/openworm/openworm; cd openworm
  3. Optional: Run ./build.sh (or build.cmd on Windows). If you skip this step, it will download the latest released Docker image from the OpenWorm Docker hub.
  4. Run ./run.sh (or run.cmd on Windows).
  5. About 5-10 minutes of output will display on the screen as the steps run.
  6. The simulation will end. Run stop.sh (stop.cmd on Windows) on your system to clean up the running container.
  7. Inspect the output in the output directory on your local machine.

Advanced

Arguments

  • -d [num] : Use to modify the duration of the simulation in milliseconds. Default is 15. Use 5000 to run for time to make the full movie above (i.e. 5 seconds).

Other things to try

  • Open a terminal and run ./run-shell-only.sh (or run-shell-only.cmd on Windows). This will let you log into the container before it has run master_openworm.py. From here you can inspect the internals of the various checked out code bases and installed systems and modify things. Afterwards you'll still need to run ./stop.sh to clean up.
  • If you wish to modify what gets installed, you should modify Dockerfile. If you want to modify what runs, you should modify master_openworm.py. Either way you will need to run build.sh in order to rebuild the image locally. Afterwards you can run normally.

FAQ

What is the Docker container?

The Docker container is a self-contained environment in which you can run OpenWorm simulations. It's fully set up to get you started by following the steps above. At the moment, it runs simulations and produces visualizations for you, but these visualizations must be viewed outside of the Docker container. While you do not need to know much about Docker to use OpenWorm, if you are planning on working extensively with the platform, you may benefit from understanding some basics. Docker Curriculum is an excellent tutorial for beginners that is straightforward to work through (Sections 1 - 2.5 are plenty sufficient).

Is it possible to modify the simulation without having to run build.sh?

Yes, but it is marginally more complex. The easiest way is to modify anything in the Docker container once you are inside of it - it will work just like a bash shell. If you want to modify any code in the container, you'll need to use an editor that runs in the terminal, like nano. Once you've modified something in the container, you don't need to re-build. However, if you run stop.sh once you exit, those changes will be gone.

How do I access more data than what is already output?

The simulation by default outputs only a few figures and movies to your home system (that is, outside of the Docker container). If you want to access the entire output of the simulation, you will need to copy it from the Docker container.

For example, say you want to extract the worm motion data. This is contained in the file worm_motion_log.txt, which is found in the /home/ow/sibernetic/simulations/[SPECIFIC_TIMESTAMPED_DIRECTORY]/worm_motion_log.txt. The directory [SPECIFIC_TIMESTAMPED_DIRECTORY] will have a name like C2_FW_2018_02-12_18-36-32, and its name can be found by checking the output directory. This is actually the main output directory for the simulation, and contains all output, including cell modelling and worm movement.

Once the simulation ends and you exit the container with exit, but before you run stop.sh, run the following command from the openworm-docker-master folder:

docker cp openworm:/home/ow/sibernetic/simulations/[SPECIFIC_TIMESTAMPED_DIRECTORY]/worm_motion_log.txt ./worm_motion_log.txt

This will copy the file from the Docker container, whose default name is openworm. It is crucial that you do not run stop.sh before trying to get your data out (see below)

What is the difference between exit and stop.sh?

When you are in the Docker Container openworm, and are done interacting with it, you type exit to return to your system's shell. This stops execution of anything in the container, and that container's status is now Exited. If you try to re-start the process using run-shell-only.sh, you will get an error saying that the container already exists. You can choose, at this point, to run stop.sh. Doing so will remove the container and any files associated with it, allowing you to run a new simulation. However, if you don't want to remove that container, you will instead want to re-enter it.

How do I enter a container I just exited?

If you run stop.sh you'll delete your data and reset the container for a new run. If, however, you don't want to do that, you can re-enter the Docker container like this:

docker start openworm                 # Restarts the container
docker exec -it openworm /bin/bash    # Runs bash inside the container

This tells Docker to start the container, to execute commands (exec) with an interactive, tty (-it) bash (bash) shell in the container openworm.

You'll be able to interact with the container as before.

Documentation

to find out more about OpenWorm, please see the documentation at http://docs.openworm.org or join us on Slack.

This repository also contains project-wide tracking via high-level issues and milestones.

org.geppetto's People

Contributors

adrianq avatar ddelpiano avatar domenic avatar gidili avatar gitter-badger avatar jrieke avatar jrmartin avatar mlolson avatar msasinski avatar mschuene avatar pgleeson avatar psj258 avatar robbie1977 avatar slarson avatar tarelli avatar waffle-iron avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

org.geppetto's Issues

The folder repository/usr is empty after launching the script

The folder repository/usr is empty after launching the script, starting the server fails for this reason:

[2012-11-22 17:13:26.293] fs-watcher Hot deploy failed for file 'owsefull.plan'. org.eclipse.virgo.nano.deployer.api.core.DeploymentException: Deployment of plan 'multi-artifact.plan' version '1.0.0' in scope 'null' failed: bundle 'org.openworm.simulationengine.core' in version range '[1.0.0, oo)' not found

Review Sprint 22

  • Review popup tutorial
  • Review GitHub maven repository
  • Bring in latest changes from development (should include the popup tutorial)
  • Create pull request to merge everything to development
  • Remove data toggle from help button
  • Make sure the create a widget tutorial is up to date
  • Loading a new simulation should also clean the Popup widgets (should we make the code generic so that any widget gets automatically cleaned?)
  • Popover should always be in front of everything (if you load a simulation which has widgets they will be behind them)
  • Fix javascript errors due to popup widgets getting notified of updated but not having the update method
  • Move popovers for the selection a simulation sample below the dropdown instead of its right
  • Move popovers for the navigation control to the right of the controls instead of below
  • Share popover doesn't move when clicking the share button (the console one does)
  • Make the zzz are you still there timer longer (1min?) It shows afters 5 min, and then one minute after that it disconnects. No code has changed since it was created.
  • Is there a gap on the welcome dialog on the right? No gap found, there's a border around the whole dialog
  • Geppetto is ready message not appearing anymore?
  • The popover for the console sometimes is displaced and it appears where the zoom buttons are
  • For some steps of the tutorial like start it asks the user to press continue while the event on the start button could be detected to go to the next step

Create samples which uses multiple entities

  • Add one sample which has two entities (for instance two purkinje cells) and specifies their position
  • Implement code to read the position of an entity off the simulation file
  • Add a more complex sample which has a simple hierarchy of nested entities

Create configuration file from worm model

This is about creating a Sibernetics Geppetto configuration file from the worm model in sibernetics.

  • Get Configuration in text format From Sibernetic
  • Update configuration generator if needed
  • Generate a worm.xml according to Geppetto SPH Schema for the worm

Test membranePorting branch with the latest

  • Take the latest of development of every bundle
  • Add testMembranesSceneWithLiquid to use cube_with_membranes_cube_water_inside
  • Try running samples from the GUI and see if they behave as they used to (no explosions)
  • Fix testMembranesSceneWithLiquid
  • Run all unit tests and report here what is passing and what is failing

Fix Console Scrollbars in Firefox

We are currently using a plugin to create our console in Geppetto, http://josscrowcroft.github.io/javascript-sandbox-console/ .

This is working fine in Chrome and all other webkit browser, but is not styling the scrollbars for the console in Firefox and they become visible as seen in the picture.
screen_shot_2014-03-22_at_17 09 45
Ideally the vertical scrollbar would be transparent, and the horizontal one should be invisible.
chromescrollbars

Due a bug reported in the Mozilla site https://bugzilla.mozilla.org/show_bug.cgi?id=77790, we can't style scrollbars in Firefox. A possibility to solve this will be to use jquery scroll pane

Allow entities selection on the frontend

Using the test scenes developed in #48

  • Allow for each of the entities to be selected on the frontend
  • Clicking on an entity results in a command being sent to console Simulation.selectEntity("entityParent.entityChild") where entityParent.entityChild is the instance path of the entity we clicked on
  • Selected entities have a different color
  • After the first sample is working design meeting to go in depth into how to deal with nested entities

The PAR doesn't work

Gives the following error

<HD0002E> Hot deploy failed for file 'owsefull.plan'. org.eclipse.virgo.nano.deployer.api.core.DeploymentException: Deployment of plan 'multi-artifact.plan'    version '1.0.0' in scope 'null' failed: bundle 'org.openworm.simulationengine.core' in version range '[1.0.0, oo)' not found

Review Sprint 23

  • Convert simulation tutorial in rst and add to docs.geppetto.org (use markedup XML not images)
  • Put back link to tutorial in Simulation dialog
  • Make sure the widget tutorial is up to date
  • Update default template for custom simulation

Ensure variable watch time consistency

If variables are added at different times to variable watch the updates sent from the server still need to be consistent, i.e. to belong to the same timestep for all variables regardless of when they were added.

  • Solver doesn't update list of new variables sent to simulation, only creates them once and reuses that same list to send the updates even after new ones have been added.
  • JLems also doesn't update list of new variables sent to simulation after adding second addwatchvariable command
  • Fix
  • Test with testbackend, neuronal and sph
  • Write qunit test

Y-axis should autoscale

Y-axis on the chart widget is not set to autoscale.
Currently the top value of the hhcell.electrical.hhpop[0].spiking plot is not visible.

gp_autoscale

Originally filed by @msasinski

Transferred here from the main OpenWorm board in this issue

Review new simulation format

  • Review simulation format
  • Write a tutorial that goes through how to create a custom simulation file explaining its structure and the different fields
  • Add question mark button to the load simulation dialog which links to the tutorial
  • Update custom tab in the Load simulation dialog if needed

Python script needs non documented third party library

Matteos-MacBook-Pro:~ matteocantarelli$ python build_server_package.py
Traceback (most recent call last):
File "build_server_package.py", line 10, in
from fabric.api import *
ImportError: No module named fabric.api

pom doesn't build

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error building POM (may not be this project's POM).

Project ID: unknown

Reason: Could not find the model file '/tmp/tmp97KXFV/org.openworm.simulationengine.core'. for project unknown

[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.reactor.MavenExecutionException: Could not find the model file '/tmp/tmp97KXFV/org.openworm.simulationengine.core'. for project unknown
at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:404)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:272)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.project.ProjectBuildingException: Could not find the model file '/tmp/tmp97KXFV/org.openworm.simulationengine.core'. for project unknown
at org.apache.maven.project.DefaultMavenProjectBuilder.readModel(DefaultMavenProjectBuilder.java:1575)
at org.apache.maven.project.DefaultMavenProjectBuilder.buildFromSourceFileInternal(DefaultMavenProjectBuilder.java:506)
at org.apache.maven.project.DefaultMavenProjectBuilder.build(DefaultMavenProjectBuilder.java:200)
at org.apache.maven.DefaultMaven.getProject(DefaultMaven.java:604)
at org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:487)
at org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:560)
at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:391)
... 12 more
Caused by: java.io.FileNotFoundException: /tmp/tmp97KXFV/org.openworm.simulationengine.core (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(FileInputStream.java:137)
at hidden.org.codehaus.plexus.util.xml.XmlReader.(XmlReader.java:124)
at hidden.org.codehaus.plexus.util.xml.XmlStreamReader.(XmlStreamReader.java:67)
at hidden.org.codehaus.plexus.util.ReaderFactory.newXmlReader(ReaderFactory.java:118)
at org.apache.maven.project.DefaultMavenProjectBuilder.readModel(DefaultMavenProjectBuilder.java:1570)
... 18 more
[INFO] ------------------------------------------------------------------------
[INFO] Total time: < 1 second
[INFO] Finished at: Thu Nov 22 16:04:37 UTC 2012
[INFO] Final Memory: 2M/111M
[INFO] ------------------------------------------------------------------------

Review and test replay branch

  • Review changes across all bundles
  • Test with all samples and test backend
  • Extensive testing of all features
  • Make sure all qunit tests pass

Note: this is targeted for release 0.1.2 so it should have higher priorities than the rest

Integrate 3D plotting widget with Geppetto

  • Setup development environment
  • Test current functionality
  • Make sure the widget architecture is updated with the latest one specified in Geppetto
  • Connect the proof of concept widget to Geppetto

Review Sprint 24

  • Improve Simulation file description
  • Make look and feel of custom help button consistent with the arrows icon

Compression of data transfer between server and client

During the resolution of #8 @tarelli @gidili and I realised that large SPH scenes result in a communications bottleneck. We decided that simulation information sent from server to client needs to be compressed in some way. I'll start the ball rolling with some ideas of how to do this:

  1. Binary not JSON
  2. Data compression
  3. Select "rendering particles" at configuration stage - only these are sent, the others are used in calculation only.
  4. Work out 3D Convex hulls of particles and send only those
  5. Only send particles which have moved by a significant distance. The server will need to keep track of what has been displayed on the client.

Make Parent WidgetController Class

Refactor out Widget Controller functionality. as suggested before; "There is some functionality in e.g. the plot controller pertaining to the creation of a plot instance, management of a list of instances, removal... Those look common to all widgets, those should inherit from an abstract WidgetController".

Add Simulator to replay recording

  • Add a Simulator which can be used to read a recording and create a Geppetto state tree
  • Write unit tests to check the created state tree from recorded simulation
  • Integrate with the rest of Geppetto
  • Create Geppetto sample scene with recording
  • Test with frontend

Move over remaining issues and wiki pages

There are still some Geppetto issues hanging in the OpenWorm repo, as well as some wiki pages, move them over to org.geppetto or subrepo.

  • Move issues
  • Copy wiki pages over
  • Update wiki links
  • Delete old wiki pages

Tutorial improvements

  • Add button to start the tutorial in the help dialog
  • Show a confirmation dialog informing the user they will be abandoning their current simulation if they decide to load the tutorial
  • Add button to quit the tutorial at any time

Add array API

It would be great to use Geppetto for basic data analysis / descriptive statistics, without having to embed trivialities (such as calculating an average) in the model description itself. In other words, at the moment it is possible to simulate a model and produce timeseries ([ti, xi] pairs) to be plotted, but what if we would like to calculate/plot functions of x (examples: calculate mean(x) or plot a histogram of cos(x)). Having an (numpy/matlab/R/scilab/...) arraylike data structure, with builtin support for arraywise base mathematical functions and indexing would be fantastic -- and eliminate the need of firing scipy/matlab/etc just to perform trivial exploratory tasks.

Random wishlist -- given an array x with simulation results

    plot(abs(x[-1000:end])) //intuitive indexing, get the last 1000 points
    histogram((x-mean(x))/10)//basic statistics and operations involving arrays/scalars

Release 0.1.2

  • Upgrade version
  • Test development branch
  • Merge development into master
  • Create bundle
  • Smoke test bundle
  • Write release notes
  • Release
  • Send release email

Lighting up a neuron issue with some colors

Depending on the color of the mesh lighting up causes the mesh to change color first then to go white and the to go back to the original color as opposed of simply going from the original color to white.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.