neurovault / neurovault_puppet Goto Github PK

Puppet installation and configuration for Neurovault

Ruby 8.57% Pascal 11.96% Shell 21.22% Puppet 56.30% Python 1.36% HTML 0.59%

neurovault_puppet's Introduction

Puppet module for installing NeuroVault

This module is intended for Puppet installation of NeuroVault, a Python web application for storing, sharing and viewing unthresholded brain activation maps.

To automatically provision and deploy a development environment with Puppet and Vagrant, see:

INSTALL_DEVSERVER.md

To deploy a production installation with Puppet standalone, see:

INSTALL_PRODUCTION.md

For troubleshooting install, see:

TROUBLESHOOTING.md

######Additional Info: For information about using your VM with an X-windows GUI, see here. For information regarding manual installation of Python environment, see see here.

neurovault_puppet's People

Contributors

Watchers

Forkers

chrisgorgo jbwexler burnash

neurovault_puppet's Issues

Add path to pycortex webgl resources to documentation

I think maybe the install should mention changing this path in the neurovault settings.py:

Pycortex static data is deployed by collectstatic at build time.
STATICFILES_DIRS = (
('pycortex-resources', '/path/to/pycortex/cortex/webgl/resources'),
('pycortex-ctmcache', os.path.join(PYCORTEX_DATASTORE,'db/fsaverage/cache')),
)

I found it like this:
$python

import cortex
cortex.file
'/usr/local/lib/python2.7/dist-packages/cortex/init.pyc'

That seemed to fix (most) of it, and the files and pycortex directories were created:

/opt/nv-env/NeuroVault/neurovault$ ls private_media/images/2/2mm16_zstat1_1_pycortex/
data
fsaverage_[inflated]mg2_9['rois'].ctm
fsaverage_[inflated]mg2_9['rois'].json
fsaverage_[inflated]mg2_9['rois'].svg

But I am missing an index.html! Page not found:

http://localhost:8000/media/images/2/2mm16_zstat1_1_pycortex/index.html

I was going to dig into the code to figure out why there is no index, but I don't know pycortex well (and I assume you guys do) so it might be 10X faster to just ask. If not, I'll see if I can figure it out!

Run portion of puppet?

Is there a way to run just a portion of the install from main? My install failed with getting the pycortex data:

==> default: Debug: Executing 'svn export --force https://github.com/NeuroVault/neurovault_data/trunk/pycortex_datastore /opt/pycortex_data'

Likely because of just internet connectivity. The remaining errors were largely warnings and notices about the dependency, but I can find a bunch of commands that were skipped, eg:

==> default: Warning: /Stage[main]/Main/Neurovault::Main[nvault-install]/Neurovault::Pycortex[install_pycortex]/Exec[chown_datastore]: Skipping because of failed dependencies

And they are substantial because manage.py runserver can't even find django (and I did source the venv).

I was going through each one and looking in the main.pp file and then the appropriate file under manifests to run manually, but I think it would be easier to just trigger the install from some point forward. I did install all of freesurfer so I'm less wanting to restart the whole thing to see if I get the same timeout!

Seamless!

I just wanted to comment that I just had THE FIRST EVER completely successful install of the NeuroVault environment from puppet... ever!! This is diabolical.

Error: Duplicate parameter 'system_user'

I got the following error when trying the vagrant dev setup

filo at MathBook Pro in ~/vms/neurovault
$ vagrant up
/Applications/Vagrant/bin/../embedded/gems/gems/vagrant-1.6.5/lib/vagrant/pre-rubygems.rb:31: warning: Insecure world writable dir /Developer/NVIDIA/CUDA-6.0/bin in PATH, mode 040777
/Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/runtime.rb:222: warning: Insecure world writable dir /Developer/NVIDIA/CUDA-6.0/bin in PATH, mode 040777
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'puppetlabs/ubuntu-14.04-64-puppet'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'puppetlabs/ubuntu-14.04-64-puppet' is up to date...
==> default: Setting the name of the VM: neurovault_default_1417457902651_52090
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 80 => 8000 (adapter 1)
    default: 8080 => 8001 (adapter 1)
    default: 22 => 2222 (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection timeout. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Rsyncing folder: /Users/filo/vms/neurovault/neurovault_puppet/confs/ => /tmp/vagrant-puppet-3/manifests
==> default: Mounting shared folders...
    default: /vagrant => /Users/filo/vms/neurovault
    default: /opt/nv_env => /Users/filo/vms/neurovault/nv_env
    default: /opt/image_data => /Users/filo/vms/neurovault/image_data
    default: /opt/pycortex_data => /Users/filo/vms/neurovault/pycortex_data
    default: /etc/puppet/modules/neurovault => /Users/filo/vms/neurovault/neurovault_puppet
==> default: Running provisioner: shell...
    default: Running: inline script
==> default: stdin: is not a tty
==> default: Reading package lists...
==> default: Building dependency tree...
==> default:
==> default: Reading state information...
==> default: The following extra packages will be installed:
==> default:   git-man liberror-perl patch
==> default: Suggested packages:
==> default:   git-daemon-run git-daemon-sysvinit git-doc git-el git-email git-gui gitk
==> default:   gitweb git-arch git-bzr git-cvs git-mediawiki git-svn diffutils-doc
==> default: The following NEW packages will be installed:
==> default:   git git-man liberror-perl patch
==> default: 0 upgraded, 4 newly installed, 0 to remove and 88 not upgraded.
==> default: Need to get 3,359 kB of archives.
==> default: After this operation, 21.8 MB of additional disk space will be used.
==> default: Get:1 http://us.archive.ubuntu.com/ubuntu/ trusty/main liberror-perl all 0.17-1.1 [21.1 kB]
==> default: Get:2 http://us.archive.ubuntu.com/ubuntu/ trusty/main git-man all 1:1.9.1-1 [698 kB]
==> default: Get:3 http://us.archive.ubuntu.com/ubuntu/ trusty/main git amd64 1:1.9.1-1 [2,555 kB]
==> default: Get:4 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main patch amd64 2.7.1-4ubuntu1 [84.4 kB]
==> default: dpkg-preconfigure: unable to re-open stdin: No such file or directory
==> default: Fetched 3,359 kB in 6s (509 kB/s)
==> default: Selecting previously unselected package liberror-perl.
==> default: (Reading database ... 61228 files and directories currently installed.)
==> default: Preparing to unpack .../liberror-perl_0.17-1.1_all.deb ...
==> default: Unpacking liberror-perl (0.17-1.1) ...
==> default: Selecting previously unselected package git-man.
==> default: Preparing to unpack .../git-man_1%3a1.9.1-1_all.deb ...
==> default: Unpacking git-man (1:1.9.1-1) ...
==> default: Selecting previously unselected package git.
==> default: Preparing to unpack .../git_1%3a1.9.1-1_amd64.deb ...
==> default: Unpacking git (1:1.9.1-1) ...
==> default: Selecting previously unselected package patch.
==> default: Preparing to unpack .../patch_2.7.1-4ubuntu1_amd64.deb ...
==> default: Unpacking patch (2.7.1-4ubuntu1) ...
==> default: Processing triggers for man-db (2.6.7.1-1) ...
==> default: Setting up liberror-perl (0.17-1.1) ...
==> default: Setting up git-man (1:1.9.1-1) ...
==> default: Setting up git (1:1.9.1-1) ...
==> default: Setting up patch (2.7.1-4ubuntu1) ...
==> default: Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
==> default:    (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1117:in `issue_deprecation_warning')
==> default: Notice: Preparing to install into /etc/puppet/modules ...
==> default: Notice: Downloading from https://forgeapi.puppetlabs.com ...
==> default: Notice: Installing -- do not interrupt ...
==> default: /etc/puppet/modules
==> default: └─┬ puppetlabs-postgresql (v4.1.0)
==> default:   ├── puppetlabs-apt (v1.7.0)
==> default:   ├── puppetlabs-concat (v1.1.2)
==> default:   └── puppetlabs-stdlib (v4.4.0)
==> default: Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
==> default:    (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1117:in `issue_deprecation_warning')
==> default: Notice: Preparing to install into /etc/puppet/modules ...
==> default: Notice: Module puppetlabs/stdlib 4.4.0 is already installed.
==> default: Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
==> default:    (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1117:in `issue_deprecation_warning')
==> default: Notice: Preparing to install into /etc/puppet/modules ...
==> default: Notice: Downloading from https://forgeapi.puppetlabs.com ...
==> default: Notice: Installing -- do not interrupt ...
==> default: /etc/puppet/modules
==> default: └─┬ puppetlabs-apache (v1.2.0)
==> default:   ├── puppetlabs-concat (v1.1.2)
==> default:   └── puppetlabs-stdlib (v4.4.0)
==> default: Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
==> default:    (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1117:in `issue_deprecation_warning')
==> default: Notice: Preparing to install into /etc/puppet/modules ...
==> default: Notice: Downloading from https://forgeapi.puppetlabs.com ...
==> default: Notice: Installing -- do not interrupt ...
==> default: /etc/puppet/modules
==> default: └─┬ jfryman-nginx (v0.2.1)
==> default:   ├── puppetlabs-apt (v1.7.0)
==> default:   ├── puppetlabs-concat (v1.1.2)
==> default:   └── puppetlabs-stdlib (v4.4.0)
==> default: Cloning into '/etc/puppet/modules/uwsgi'...
==> default: Cloning into '/etc/puppet/modules/python'...
==> default: Running provisioner: puppet...
==> default: Running Puppet with nvault.pp...
==> default: stdin: is not a tty
==> default: Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
==> default:    (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1117:in `issue_deprecation_warning')
==> default: Debug: Runtime environment: puppet_version=3.7.1, ruby_version=1.9.3, run_mode=user, default_encoding=UTF-8
==> default: Info: Loading facts
==> default: Debug: Loading facts from /etc/puppet/modules/apt/lib/facter/apt_updates.rb
==> default: Debug: Loading facts from /etc/puppet/modules/apt/lib/facter/apt_update_last_success.rb
==> default: Info: Loading facts
==> default: Debug: Loading facts from /etc/puppet/modules/concat/lib/facter/concat_basedir.rb
==> default: Info: Loading facts
==> default: Debug: Loading facts from /etc/puppet/modules/python/lib/facter/python_version.rb
==> default: Debug: Loading facts from /etc/puppet/modules/python/lib/facter/virtualenv_version.rb
==> default: Debug: Loading facts from /etc/puppet/modules/python/lib/facter/pip_version.rb
==> default: Info: Loading facts
==> default: Debug: Loading facts from /etc/puppet/modules/stdlib/lib/facter/puppet_vardir.rb
==> default: Debug: Loading facts from /etc/puppet/modules/stdlib/lib/facter/pe_version.rb
==> default: Debug: Loading facts from /etc/puppet/modules/stdlib/lib/facter/facter_dot_d.rb
==> default: Debug: Loading facts from /etc/puppet/modules/stdlib/lib/facter/root_home.rb
==> default: Debug: Puppet::Type::Package::ProviderRpm: file rpm does not exist
==> default: Debug: Puppet::Type::Package::ProviderUp2date: file /usr/sbin/up2date-nox does not exist
==> default: Debug: Puppet::Type::Package::ProviderPacman: file /usr/bin/pacman does not exist
==> default: Debug: Puppet::Type::Package::ProviderSun: file /usr/bin/pkginfo does not exist
==> default: Debug: Puppet::Type::Package::ProviderSunfreeware: file pkg-get does not exist
==> default: Debug: Puppet::Type::Package::ProviderAix: file /usr/bin/lslpp does not exist
==> default: Debug: Puppet::Type::Package::ProviderNim: file /usr/sbin/nimclient does not exist
==> default: Debug: Puppet::Type::Package::ProviderOpenbsd: file pkg_info does not exist
==> default: Debug: Puppet::Type::Package::ProviderFreebsd: file /usr/sbin/pkg_info does not exist
==> default: Debug: Puppet::Type::Package::ProviderHpux: file /usr/sbin/swinstall does not exist
==> default: Debug: Puppet::Type::Package::ProviderUrpmi: file urpmi does not exist
==> default: Debug: Puppet::Type::Package::ProviderPortage: file /usr/bin/emerge does not exist
==> default: Debug: Puppet::Type::Package::ProviderZypper: file /usr/bin/zypper does not exist
==> default: Debug: Puppet::Type::Package::ProviderPortupgrade: file /usr/local/sbin/portupgrade does not exist
==> default: Debug: Puppet::Type::Package::ProviderFink: file /sw/bin/fink does not exist
==> default: Debug: Puppet::Type::Package::ProviderYum: file yum does not exist
==> default: Debug: Puppet::Type::Package::ProviderPkg: file /usr/bin/pkg does not exist
==> default: Debug: Puppet::Type::Package::ProviderPkgin: file pkgin does not exist
==> default: Debug: Puppet::Type::Package::ProviderPorts: file /usr/local/sbin/portupgrade does not exist
==> default: Debug: Puppet::Type::Package::ProviderAptrpm: file rpm does not exist
==> default: Debug: Puppet::Type::Package::ProviderOpkg: file opkg does not exist
==> default: Debug: Puppet::Type::Package::ProviderRug: file /usr/bin/rug does not exist
==> default: Debug: /Package[python-pip]: Provider apt does not support features virtual_packages; not managing attribute allow_virtual
==> default: Debug: /Package[virtualenv]: Provider apt does not support features virtual_packages; not managing attribute allow_virtual
==> default: Debug: /Package[python]: Provider apt does not support features virtual_packages; not managing attribute allow_virtual
==> default: Debug: Executing '/usr/bin/dpkg-query -W --showformat '${Status} ${Package} ${Version}\n' python'
==> default: Debug: importing '/etc/puppet/modules/neurovault/manifests/main.pp' in environment production
==> default: Debug: Automatically imported neurovault::main from neurovault/main into production
==> default: Error: Duplicate parameter 'system_user' for on Neurovault::Main[nvault-install] at /tmp/vagrant-puppet-3/manifests/nvault.pp:145 on node localhost.stanford.edu
==> default: Error: Duplicate parameter 'system_user' for on Neurovault::Main[nvault-install] at /tmp/vagrant-puppet-3/manifests/nvault.pp:145 on node localhost.stanford.edu
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

Use settings_local.py instead of settings.py

Instead of modifying settings.py add and overwrite variable in settings_local.py to avoid accidental commits.

neurovault DB user should be able to create databases

This should fix it:

sudo -u postgres psql -c "ALTER USER neurovault CREATEDB"

Otherwise a test database cannot be created and no tests can be run.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.