Other commands return error messages like: Unknown application: myapp, but logscommand doesn't, making it look like the command succeeded.

Already exists some open source paas like tsuru (https://github.com/globocom/tsuru) what motivate you to create another paas?

Need a way to reliably destroy ZFS containers when there are child dependencies.

ubuntu@ip-10-60-86-207:~$ sudo lxc-destroy -n myapp-s5
cannot destroy 'tank/myapp-s5': filesystem has children
use '-r' to destroy the following datasets:
tank/myapp-s5@myapp-s5_v2_web_10002
tank/myapp-s5@myapp-s5_v3_web_10001
lxc_container: Error destroying rootfs for myapp-s5
Destroying myapp-s5 failed
ubuntu@ip-10-60-86-207:~$

Calling shipbuilder get:remove VARIABLE -aAPP is not properly removing the variable from the environment. For ex:

$ shipbuilder config:get POSTGRES_ENGINE -aAPP
Client connecting via 'ubuntu@APP..
django_postgrespool

$ shipbuilder config:remove POSTGRES_ENGINE -aAPP
Client connecting via 'ubuntu@APP..
0:01 === Removing environment variables..
0:01
Finished removing environment variables.
0:01 NOTICE: Redeploy deferred, changes will not be active until next deploy is triggered

Notice that no environment variables were removed.
Additionally, I don't believe that this should have been deferred.

When attempting to create a new node like so:

./install/node.sh -H sb-node7a -d /dev/xvdb -s /dev/xvdc install

I get the following error:

info: installLxc() succeeded
info: attempting to unmount /mnt and /dev/xvdb to be safe
info: existing fs type on /dev/xvdb is ext4
info: formatting /dev/xvdb with zfs
invalid vdev specification
use '-f' to override the following errors:
/dev/xvdb is part of potentially active pool 'tank'
error: command 'sudo zpool create -o ashift=12 tank /dev/xvdb' exited with non-zero status 1
error: remote prepareNode() invocation exited with non-zero status 1

If I don't specify the swap device, i get a different error:

./install/node.sh -H sb-node7a -d /dev/xvdb install
info: auto-detected shipbuilder host: ubuntu@sb
info: auto-detected lxc filesystem: zfs
info: auto-detected zfs pool: tank
info: verifying ssh and sudo access for 2 hosts
info: testing host ubuntu@sb .. succeeded
info: testing host sb-node7a .. succeeded
....
sent 80 bytes received 20 bytes 28.57 bytes/sec
total size is 31289 speedup is 312.89
error: prepareNode() missing required zfs parameter: $zfsPool
error: remote prepareNode() invocation exited with non-zero status 1

The relevant env nodes are:

$ cat env/LXC_FS
zfs
brock ~/dev/sendhub/shipbuilder [master*]
$ cat env/ZFS_POOL
tank

Any ideas?

ShipBuilder no longer compiles. Appears to require Go upgrade.

Service on all container nodes which periodically syncs the iptables to match the currently listed containers.

• Upon finding a mis-match, emits warning messge to log.
• Automatically corrects the port situation.

This would be great because sometimes the iptables rules mysteriously get out of sync for the running containers on the system, leading to this emerging as a habitual pattern for LB<=>Container communication failure.

Wondering if you have any suggestions on the following.

I recently new nodes on our installation based on the ami build-nd-5. Upon initial deploy to them, they appear fine but after some period of time they no longer properly report their IP back to the shipbuilder server:

..
3:46 [sb-node7a] [s1_v473_egress_10002] - error retrieving ip
3:46 Retrying starting app dyno egress on host sb-node7a, failure reason: exit status 1
..

After the deploy, when inspecting the dynos on the node, everything looks fine. The dyno is running and I haven't found any problems with it with the exception that the shipbuilder server doesn't get the ip and thus doesn't update iptables/the loadbalancer to send traffic to it.

If you have any suggestions, it would be much appreciated.

Would be cool if you could add help to the command line and offer more options in terms of node management

shipbuilder -h or --help

shipbuilder list (shows running containers)

Scaling to web=0 after first deploy puts app in a weird state where it won't allow you to scale back up afterwards. It claims the app hasn't had a first deploy.

Destroying an app only removes it from the config and destroy build box containers. It should also shutdown ALL dynos running on nodes, including failed deploys etc.. and then ultimately remove the base app image globally.

ShipBuilder

Additional information is available at https://shipbuilder.gigawatt.io

The link goes to https://github.com/jaytaylor/shipbuilder/blob/master/shipbuilder.gigawatt.io rather than shipbuilder.gigawatt.io (where it should go).

I would correct it but I don't have capaicty right now.

Cheers
Dean

After a number of failed builds, I ran apps:destroy -a{name} and the command ran successfully, but running logs -a{name} still shows a continuos stream of network events and app output. logs is the only command that works, any other command will return Unknown application error.

So, the main problem you're going to run into with this setup is that Upstart has syscall hook based PID-tracking which it uses to keep track of the processes it is supposed to manage during the launch process. Since you are using start-stop-daemon to run /app/run-in-context, then using that to run /app/runs, then running multiple PIDs from /app/run, this is three layers of PID indirection which will confuse upstart greatly and cause it to get mixed up when it sends signals to downstream processes and potentially create a number of unresponsive, unreliable, and/or orphan PIDs.

This setup could be made more reliable by making the /app/run be called from Upstart directly without start-stop-daemon, then place a signal handler into /app/run which passes any signals it receives down to every child process it owns. An easy way of doing that can be, if your pid happened to be 12345, you can pass the signal to -12345 to send it to your entire POSIX Process Group. Or you can keep appending the $$ value to some list in bash, then loop across the list in the handler to signal all subprocesses using the signal received from above. Another option is to make a pre-stop hook which sends the signals where they are needed before the stop hook stops the master process.

Another option: make a separate upstart config for each daemon in the Procfile, so Upstart can watch each of them. Then use some of Upstart's dependencies, to make a task which depends on all the others to perform the starts and stops, while Upstart will be able to see each PID. Then, in the bash launcher of each task, use the exec function to launch the tasks, which causes them to inherit bash's PID. Then the PID it runs with and the PID Upstart expects are the same, and you get reliable operation.