mykafkabenchmark's Introduction
Python Fabric based Streaming OODT install. --------- OPERATION --------- - Ensure passkey access to deploying nodes - Setup config file. -- A template of which is found in ./config_template.ini All parameters found here must be set, comment out sections for aplication which you do not wish to have installed. -- Where greater configuration of applications is desired than is made available via this config file, templates contains application-specific config file templates. These may be edited. Any changes made here will propigate to all nodes receiving this file. - Ensure 5GB RAM available for Tachyon - Ensure the listed dependancise are installed on the machine from which this install application will be run. - Where build commands are to be issued the node on which this script is called must be of the same architecture as deploy nodes - To run: Install: $ python deploy install configs/config.ini Clean: $ python deploy clean configs/config.ini - Log files are found in ./logs currently. Edit the logging setup in deploy for changes in fabricsoodt reporting, all loggers are named as subsidary to fabricsoodt' ------------ DEPENDANCIES ------------ Python 3 python-dev python-pip pip modules*: pystache configparser yarn scala maven 2.x+ openjdk-7-jdk *Use < pip install module-name --user > to install in user space (no sudo required) $ pip install -r requirements.txt $ cd DeploymentSystems/Yarn $ python setup.py install Hint: for colour logs install coloredlogs $ pip install coloredlogs And run with -cl or --colouredLogs flag Some optional applications have their own dependencies. Given that this install system is intended to not require sudo the user is expected to ensure the following are installed. If sudo access is available set this in the config script and attempts to auto install will be made Mesos: - build-essentials - python-dev - python-boto - libcurl4-nss-dev - maven (*Only required for Mesos 0.18.1 or newer*) - libsvn-dev (*Only required for Mesos 0.21.0 or newer) - libsasl2-dev (*Only required for Mesos 0.14.0 or newer*) - libapr1-dev (*Only required for Mesos 0.21.0 or newer*) ---------------------------- Centos specific instructions ---------------------------- - To install pip with sudo: - Add EPEL repository sudo pm -iUvh http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm sudo yum -y update sudo yum -y install python-pip - Or to install pip locally without sudo: - curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py" - python get-pip.py --user
mykafkabenchmark's People
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.