Giter Club home page Giter Club logo

azk's Introduction

A development environment orchestration tool

azk lets developers easily and quickly install and configure their development environments. Period.

Usage of azk

Quick start

Installing

$ curl -Ls http://azk.io/install.sh | bash

Requirements:

For further details, please see the docs.

Using azk

Starting a new project

If you are starting a new application project, you can already use azk to obtain the proper runtime as well the corresponding generators for your chosen language and then generate the application's basic structure. An example in Node.js would look like this:

$ cd ~/projects
$ azk shell --image azukiapp/node # obtaining the runtime
    # mkdir app-name
    # npm init                    # building the application's basic structure
    ...
    # exit
$ cd app-name
$ azk init
azk: `node` system was detected at 'app-name'
azk: 'Azkfile.js' generated

$ azk start

Using azk with an existing project

When you have an application project that's already started, and want to use azk to streamline its development environment, all you have to do is:

$ cd [my_application_folder]
$ azk init
azk: 'Azkfile.js' generated
...
$ azk start

Main features

  • Multiplatform: Works both on Linux & Mac OS X (requires 64-bit platform);
    • Windows planned. Want azk to run in Windows? Thumbs up here: #334
  • Images: via azk images, Docker Registry or run your own Dockerfile;
  • Built-in load-balancer;
  • Built-in file sync;
  • Automatic start-up (and reload) script;
  • Logging;
  • And simple and easy to use DSL to describe systems architecture;

Documentation

You can find our documentation online at: http://docs.azk.io/

Run Project button

Clicking the Run Project button (or "azk button") on a GitHub repo is the best way to quickly and safely run its code on your local machine.

Run project

To add a Run Project button to a repo, you'll just need to add an Azkfile.js to the project and put the following badge in your README.md file (the following example is for a hypothetical repository with the URL https://github.com/username/repo and a branch called azkfile containing the Azkfile.js):

[![Run project](https://s3-sa-east-1.amazonaws.com/assets.azk.io/run-project.png)](http://run.azk.io/start/?repo=username/repo&ref=azkfile)

Check out the Run Project Gallery for examples of up to date forks of popular projects using it.

Deploying

After you locally run a project using Run Project button, deploying it to DigitalOcean is very simple.

First, put your personal access token into a .env file:

$ cd path/to/the/project
$ echo "DEPLOY_API_TOKEN=<YOUR-PERSONAL-ACCESS-TOKEN>" >> .env

Then, just run the following:

$ azk shell deploy

Find further instructions on how to deploy to DigitalOcean using azk here.

Basic Vocabulary

System of Systems

azk is based on the concept of System of Systems. Accordingly, applications (your code), services and workers (such as databases, webservers and queue systems) are treated as systems that communicate with each other and together make the primary system. Using this paradigm, azk installs and manages development environments. While this may seem overkill at first, it actually makes it a lot easier to manage the development and execution environments of an application (in its parts - the "systems" - or in its entirety - the full "system of systems").

Images

In order to automate the provisioning of development environments, azk uses pre-built custom images. These images follow the Docker standard and can be found in: azk images, Docker Index or Dockerfile.

Azkfile.js

Azkfile.js files are the cornerstone of how to use azk. These simple manifest files describe the systems that make your system of systems as well as the images used in their execution. They also describe parameters and execution options.

More information here.

Contributions

Check our Contributing Guide for instructions on how to help the project!

Share the love and star us here in Github!

License

"Azuki", "azk" and the Azuki logo are copyright (c) 2013-2016 Azuki Serviços de Internet LTDA.

azk source code is released under Apache 2 License.

Check LEGAL and LICENSE files for more information.

azk's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

azk's Issues

Weird behaviour using azk exec

Fail 1:

$ azk exec java -version
ized option: -v
Could not create the Java virtual machine.

Fail 2:

$ azk exec "java -version"
21 21:53:37 exec: "java -version": executable file not found in $PATH

Success:

$ azk exec -i /bin/bash
bash-4.1# java -version
java version "1.6.0_30"
OpenJDK Runtime Environment (IcedTea6 1.13.1) (rhel-3.1.13.1.el6_5-x86_64)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)

azkfile.json:

{
  // Application id
  "id": "11f32940b12d11e390e7f5de2d0db329",

  "box": "centos:6.4",

  "envs": {
    "dev": {
      // "log_dir": "./log",
      "host": "my-java-app",
      "alias": [
      ],
      "env": {
      }
    }
  },

  "build": [
    "yum install -y java-1.6.0-openjdk-devel.x86_64"
  ],

  "services": {
    // Services box depedencies
    // "postgresql": { "box": "azukiapp/postgresql" },
    // "redis": { "box": "azukiapp/redis#0.0.1" },

    "web": {
      "command": ""
    }
  }
}

System not supported

azk: System not supported (see http://azk.io)

My System:
Linux rd 3.13.0-30-generic #55-Ubuntu SMP Fri Jul 4 21:40:53 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Refactor the configuration of ports to auto select ports.

The azk uses several ports for communication of internal components:

5641   # spfs - p9fs server
5622   # ssh port to virtual machine
15680  # http balancer
5640   # rpc commands
5639   # realtime events

These ports may conflict with ports used by the user and that make the process too complicated for implementing agent.

Because these ports are just ports for communication of the components of AZK they do not necessarily need to be configured for any specific port.

Improve file sync performance

Use of the virtual machine depends on a service file sharing between the host and guest.

We are currently using the https://github.com/nuxlli/spfs, a file server based p9fs protocol, embedded directly in the azk agent.

However this performance far from feasible for consistent use of azk, it is necessary improve the performance of the spfs or find some alternative to this service.

azk exec skipping first output characters

$ azk exec ruby --version
.0p353 (2013-11-22 revision 43784) [x86_64-linux]

$ azk exec -i /bin/bash
root@2843d79fd41c:/azk/app# ruby --version
ruby 2.0.0p353 (2013-11-22 revision 43784) [x86_64-linux]

Add annotations support to containers

Today to store information about "AZK" in one container, a labels and values system is being used (by concatenating the name of the container). But this system makes it difficult to retrieve this information.

Ideally a system of notes on container would be ideal, but as it is not yet supported by Docker, alternative and improve this system of labels and add a step serialization and deserialization to create and retrieve information from the container.

Implement the command `kill`

Today if any error occurs in the execution of any command within the system of containers, the containers can happen to continue running and the user is left with no option to kill the process.

A kill command to accept as parameter the azk pid would be an option to force the kill of process.

This command must respect the --no-remove passed when creating the container.

Wait for the system to run

When commands start or scale are executed quickly return warning that the instances were created, but this does not guarantee that the system is already ready to use.

It would be important awaits the availability of instances before giving them as initialized. And just in case of a failure to show the error before finalizing.

Implement the command `update`

Today the update process azk is completely manual and follows the steps:

$ cd ~/.azk
$ azk agent stop
$ rm data/vm/debian2docker.iso
$ azk configure

The ideal is to have a command that simplify the whole process and avoid errors and report any issue that might happen in the process.

Improve Makefile to install balancer http

In the future, an image of CoreOS with http balancer already installed and configured will be made available. As of this moment, the balancer can be configured with the recipe in private/lib/balancer. However, even a step this simple could be improved with a configuration process via Makefile.

Commands that depend on azkfile.json can be optimized

Commands that depend on the azkfile.json file and the azk-agent seek and validate the azkfile.json file twice. First, at the command execution time on the host machine and then on the agent.

As nearly all azk commands must be executed within a azk-application folder, removing the first validation can improve the performance of these commands.

Add the command `docker`

Under the hood the azk makes use of the Docker management as containers system, for users who have mastered the use of the Docker power, the run Docker through 'azk' would be a great gain functionality.

For this the 'azuki docker' command could be implemented, and the principle the best way to do this is by making an alias for the command 'ssh vm azk "docker *"'

Show provision step in cli and support reprovision

The call to command start not show any information that provisioning the application is being executed, giving the idea that this start process is slow.

Também não há nenhuma opção de "forçar provisioning" para um sistema onde o provisionamento já foi executado.

Command logs

When a system is run via azk start the command may fail and in this circumstance, it would to be important be able to view the log information about this failure.

Or even in cases where the system is implemented successfully, it would be nice to view the output logs.

Implement exec command

  • Parse azkfile.json
  • Deploy app directory
  • Provision image-box
  • Provision image-app
  • Execute a command

Azk init when there is no "app"

I'm trying to create a sample azk file from a repo that do not have an app. Is just a Docker image repo. But I can't do it.

Error:

azk: TypeError: Cannot call method 'match' of undefined
    at Object.hash_key (/Users/dynaum/.azk/src/generator/index.js:71:14)
    at Object.eval (eval at <anonymous> (/Users/dynaum/.azk/node_modules/handlebars/dist/cjs/handlebars/compiler/javascript-compiler.js:181:23), <anonymous>:3:94)
    at prog (/Users/dynaum/.azk/node_modules/handlebars/dist/cjs/handlebars/runtime.js:137:15)
    at Object.<anonymous> (/Users/dynaum/.azk/node_modules/handlebars/dist/cjs/handlebars/base.js:133:23)
    at Object.eval (eval at <anonymous> (/Users/dynaum/.azk/node_modules/handlebars/dist/cjs/handlebars/compiler/javascript-compiler.js:181:23), <anonymous>:3:25)
    at Object.ret (/Users/dynaum/.azk/node_modules/handlebars/dist/cjs/handlebars/runtime.js:106:30)
    at Object.ret [as _tpl] (/Users/dynaum/.azk/node_modules/handlebars/dist/cjs/handlebars/compiler/compiler.js:459:21)
    at Object.generator.render (/Users/dynaum/.azk/src/generator/index.js:60:32)
    at Cmd.action (/Users/dynaum/.azk/src/cmds/init.js:17:14)
    at Cmd.run (/Users/dynaum/.azk/src/cli/command.js:200:16)

Error when syncing assets with s3

Fatal error: Failed to list content of bucket xpto.com
Error: The difference between the request time and the current time is too large.

Idea: use a Dockerfile as a base box

For discussion:

Imagine I have a Dockerfile file used to build my production environment. I would like to use that Dockerfile as a base box in my azkfile.json and add my development dependencies. Something like this:

{
  "box": "Dokerfile",

  "build": [
    "apt-get install -y gcc"
  ]
}

Does that make any sense?

Replace template system for dsl methods.

In Azkfile.js

mount_folders: {
  '.': '/azk/<%= system.name %>',
},
http: {
  hostname: "<%= system.name %>.<%= azk.default_doman %>"
}

Replace for:

var namespace = "example";
mount_folders: {
  '.': '/azk/::system::' ,
},
http: {
  hostname: url("::system::", namespace, default_domain)
}

Support the docker directly without a vm

Now azk require a virtual machine to provision and execute application. But in Linux it would not be necessary, instead of a virtual machine can use the system directly containers, which makes the azk faster on Linux.

To support this it is necessary to revise the agent and the use of the virtual machine.

Running a command in subfolders of the application generates an error

When you run a command like ps, service or exec in subfolders of an azk app, the azkfile.json is correctly found and the azk app folder is correctly mapped to azk-agent. However, Docker runs a command that is called to map subfolders in containers when it should call a command to map the folder corresponding to azk app in containers."

Error to install node lib: sqlite3

Reported by @dynaum:

> [email protected] install /azk/dynaum-com/node_modules/sqlite3
> node-pre-gyp install --fallback-to-build

node-pre-gyp http GET http://node-sqlite3.s3.amazonaws.com/Release/node_sqlite3-v2.2.0-node-v11-linux-x64.tar.gz
node-pre-gyp http 200 http://node-sqlite3.s3.amazonaws.com/Release/node_sqlite3-v2.2.0-node-v11-linux-x64.tar.gz
node-pre-gyp ERR! UNCAUGHT EXCEPTION
node-pre-gyp ERR! stack Error: EPERM, chown '/azk/dynaum-com/node_modules/sqlite3/lib/node_sqlite3.node'
node-pre-gyp ERR! System Linux 3.2.0-4-amd64
node-pre-gyp ERR! command "node" "/azk/dynaum-com/node_modules/sqlite3/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /azk/dynaum-com/node_modules/sqlite3
node-pre-gyp ERR! node -v v0.10.28
node-pre-gyp ERR! node-pre-gyp -v v0.2.6
node-pre-gyp ERR! This is a bug in `node-pre-gyp`.
node-pre-gyp ERR! Try to update node-pre-gyp and file an issue if it does not help:
node-pre-gyp ERR!     <https://github.com/springmeyer/node-pre-gyp/issues>
npm ERR! [email protected] install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 7
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the sqlite3 package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-pre-gyp install --fallback-to-build
npm ERR! You can get their info via:
npm ERR!     npm owner ls sqlite3
npm ERR! There is likely additional logging output above.

npm ERR! System Linux 3.2.0-4-amd64
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "install"
npm ERR! cwd /azk/dynaum-com
npm ERR! node -v v0.10.28
npm ERR! npm -v 1.4.9
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /azk/dynaum-com/npm-debug.log
npm ERR! not ok code 0

Add support to more options in `box` option

As of this moment, you can only set "github repository" in the box option. However, there should be support to more options, like a relative path or other git server repositories.

See below an example of how the syntax for this could to be:

{
...
  box: [
    "ubuntu:12.04", // Docker image
    "azukiapp/ruby-box#stable", // github
    "../ruby-box", // relative path
    "ssh://[email protected]/azukiapp/ruby-box.git"
  ],
...
}

/dev/null: Too many open files

- Scale.........system `zeus` from 0 to 1 instances...
- Checking......image `dynaum/ruby-bundler-node:latest`...
azk: Error: HTTP code is 500 which indicates error: server error - Cannot start container 657521639af6af82b571bc206aae8a4a3799d77b7929ba30f47a96171d6883d3: open /dev/null: too many open files
    at Modem.buildPayload (/Users/dynaum/.azk/node_modules/docker-modem/lib/modem.js:134:15)
    at IncomingMessage.<anonymous> (/Users/dynaum/.azk/node_modules/docker-modem/lib/modem.js:111:14)
    at IncomingMessage.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
From previous event:
    at Container._.each.newClass.(anonymous function) [as start] (/Users/dynaum/.azk/src/utils/index.js:78:39)
    at Docker.<anonymous> (/Users/dynaum/.azk/src/docker/run.js:122:20)
    at eval (eval at <anonymous> (/Users/dynaum/.azk/node_modules/traceur/src/node/traceur.js:24:17), <anonymous>:460:32)
    at Object.eval (eval at <anonymous> (/Users/dynaum/.azk/node_modules/traceur/src/node/traceur.js:24:17), <anonymous>:396:23)

Refactor the class system to be more modular

At the moment the class "System" this very monolithic, it made a lot of responsibility to stay grouped in one place and become difficult to maintain that part of the code, a refactor would be important.

In this case, the class would be divided:

  • System
  • Docker options
  • Run (start, stop, kill)
  • Scale
  • Provision
  • Upgrade README

Error on container start

Some times this error occurs when I try to run a shell command:

azk: Error: HTTP code is 500 which indicates error: server error - Cannot start container 595fe29500f6a2e83990634f5bde2090afc4f59d958aa4f83ac93ed8d19bc4b6: lstat /home/docker/files: stale NFS file handle
    at Modem.buildPayload (/Users/dynaum/.azk/node_modules/docker-modem/lib/modem.js:134:15)
    at IncomingMessage.<anonymous> (/Users/dynaum/.azk/node_modules/docker-modem/lib/modem.js:111:14)
    at IncomingMessage.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
From previous event:
    at Container._.each.newClass.(anonymous function) [as start] (/Users/dynaum/.azk/src/utils/index.js:78:39)
    at Docker.<anonymous> (/Users/dynaum/.azk/src/docker/run.js:122:20)
    at eval (eval at <anonymous> (/Users/dynaum/.azk/node_modules/traceur/src/node/traceur.js:24:17), <anonymous>:460:32)
    at Object.eval (eval at <anonymous> (/Users/dynaum/.azk/node_modules/traceur/src/node/traceur.js:24:17), <anonymous>:396:23)

Trying to make a request to an instance that I have a shell openned

I am trying to run tests on my web app that make a request to the api. But if I have a shell open with the api I receive this error:

azk: TypeError: Cannot read property 'PublicPort' of undefined
    at /Users/dynaum/.azk/src/manifest/system.js:429:65
    at Function.forEach (/Users/dynaum/.azk/node_modules/lodash/dist/lodash.js:3297:15)
    at /Users/dynaum/.azk/src/manifest/system.js:427:8
    at forOwn (/Users/dynaum/.azk/node_modules/lodash/dist/lodash.js:2105:15)
    at Function.forEach (/Users/dynaum/.azk/node_modules/lodash/dist/lodash.js:3302:9)
    at System.Object.defineProperty.value [as _dependencies_envs] (/Users/dynaum/.azk/src/manifest/system.js:426:6)
    at System.Object.defineProperty.value [as _more_envs] (/Users/dynaum/.azk/src/manifest/system.js:406:11)
    at System.<anonymous> (/Users/dynaum/.azk/src/manifest/system.js:165:31)
    at eval (eval at <anonymous> (/Users/dynaum/.azk/node_modules/traceur/src/node/traceur.js:24:17), <anonymous>:460:32)
    at Object.eval (eval at <anonymous> (/Users/dynaum/.azk/node_modules/traceur/src/node/traceur.js:24:17), <anonymous>:396:23)
From previous event:
    at /Users/dynaum/.azk/src/utils/index.js:44:17
    at node.js:902:3

Autocomplete should not display all available commands

Commands autocompleting in azk lists all azk-* available commands. However many of these are for internal use only, such as azk-image-generate or azk-provision, and should not be listed among autocomplete options.

rbenv (in which this base code was inspired) uses an internal notation in which the prefix sh- is added to commands that must be listed among autocomplete options in order to distinguish them. That may be the best solution for this issue.

Index out of range on azk exec

error:

$ azk exec uname

buffer.js:425
    throw new RangeError('index out of range');
          ^
RangeError: index out of range
    at checkOffset (buffer.js:425:11)
    at Buffer.readUInt32BE (buffer.js:494:5)
    at IncomingMessage.<anonymous> (/Users/lucasfais/.azk/node_modules/docker-modem/lib/modem.js:154:40)
    at IncomingMessage.EventEmitter.emit (events.js:101:17)
    at emitReadable_ (_stream_readable.js:418:10)
    at emitReadable (_stream_readable.js:412:7)
    at onEofChunk (_stream_readable.js:395:3)
    at readableAddChunk (_stream_readable.js:139:7)
    at IncomingMessage.Readable.push (_stream_readable.js:123:10)
    at HTTPParser.parserOnMessageComplete (_http_common.js:154:14)
    at Socket.socketOnEnd (_http_client.js:264:12)
    at Socket.EventEmitter.emit (events.js:126:20)
    at _stream_readable.js:896:16
    at process._tickCallback (node.js:664:11)

azkfile.json:

{
  // Application id
  "id": "11f32940b12d11e390e7f5de2d0db329",

  "box": "centos:6.4",

  "envs": {
    "dev": {
      // "log_dir": "./log",
      "host": "my-java-app",
      "alias": [
      ],
      "env": {
      }
    }
  },

  "build": [
    "yum install -y java-1.6.0-openjdk-devel.x86_64"
  ],

  "services": {
    // Services box depedencies
    // "postgresql": { "box": "azukiapp/postgresql" },
    // "redis": { "box": "azukiapp/redis#0.0.1" },

    "web": {
      "command": ""
    }
  }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.