aeolus-incubator / dev-tools Goto Github PK

dev-tools call db:schema:load twice

this is a bug, that did not matter until now

after adding FKs the fact that db:schema:load is called twice causes the install to fail.

This is caused by the way rails handle situation when creating a table that already exist: it calls 'drop table', but that is needed is 'drop table cascade'.

the bug is fixed here:
martinpovolny@9d5a3d9

so far I have tested rhel6.3 only and on that distro my fix works as expected

configuration of converge-ui should be replaced with new upstream ui-alchemy as needed.

One of the three conductor processes (part of #20 ).

Right now only Fedora 16/17 and RHEL 6 are supported. Allow bootstrap.sh to succeed with additional OS'es.

Right now, bootsrap.sh supports ubuntu/debian as long as RBENV_VERSION is defined. It should also work without rbenv. Additional tasks that I am going to stick under this issue, since they are closely related:

1. Update the script to check for ubuntu/debian versions and warn user if not known to work.
2. Document known working ubuntu/debian versions in the README.
3. Update ubuntu/debian package installs to be more like the fedora/el package installs (and quit if a given package did not install).

As part of #20 .

Error: Could not set 'directory' on ensure: Could not find group puppet
Error: Could not set 'directory' on ensure: Could not find group puppet
Wrapped exception:
Could not find group puppet
Error: /File[/var/lib/puppet/log]/ensure: change from absent to directory failed: Could not set 'directory' on ensure: Could not find group puppet
Debug: /File[/etc/puppet/ssl/private]/ensure: created
Debug: /File[/etc/puppet/ssl/certificate_requests]/ensure: created
Debug: Finishing transaction 25072740
Error: Got 3 failure(s) while initializing: Could not set 'directory' on ensure: Could not find group puppet; Could not set 'directory' on ensure: Could not find group puppet
Wrapped exception:
Could not find group puppet; change from absent to directory failed: Could not set 'directory' on ensure: Could not find group puppet

• '[' '!' x = x ']'

This should remove the need to sudo the gem install.

Note: it appears that Fedora/RHEL does a user-level install if the gem install is invoked by a non-root user. I believe you can force the behavior in Ubuntu/Debian like : gem install --user-install but haven't verified yet. (This should work on Fedora as well)

Update bootstrap.sh to be Mac OS X compatible.

bootstrap.sh right now is rather fragile in "system ruby" mode. The problems tend to lie in the versions of gems that may already exist on the system, for instance, the oft-mentioned puppet bug that complains if a the puppet group does not exist. There has also been an issue with newer facter gems. And, rubygem-thor on Fedora 17 causes an issue later on during bundle install.

I think the solution is just to ignore system ruby gems, rather than try to the multitude of bugs we may face across distros / gem versions. In particular, this worked well for me on Fedora 17 with the "problem gems" already installed on the system.
cwolferh/dev-tools@master...less_fragile_system_ruby2

If there isn't an objection, I'll submit a pull request after #48 gets merged (they both touch one of the same lines in bootstrap.sh, so avoiding a merge conflict).

given the imgfac commit redhat-imaging/imagefactory@7857827 , the release script should be updated : the plugins directory has been renamed from imagefactory-plugins to imagefactory_plugins

I'm not doing this myself as it will depend on the imgfac commit we will checkout from the script itself

There are a couple reasons to do this.

1. Telling folks to run scripts as root is frowned upon. :-(
2. It will simplify the script a bit, especially when it comes to rbenv/ruby installation.

The ramifications are mainly:
*Change some lines that need sudo, e.g. "yum install..." to "sudo yum install...,"
*Remove "su $DEV_USERNAME -c" prefixes.
*Move any installs out of puppet to the beginning of the script. It is OK to leave the dependency blocks in the puppet manifests as a sanity check, they just will have already been met. Note that puppet is currently invoked twice-- the first invocation installs any needed packages and the 2nd actually does the work of setting up the dev environment. This will be changed to invoking puppet just once (the first step of puppet-installing deps no longer being needed).

Right now, bootstrap.sh relies on deltacloud, image factory and image warehouse already being set up. We should support creating local upstream deltacloud and image factory instances if desired (but not image warehouse since that is going away). We may also want to support installing certain branches rather just upstream of deltacloud and image factory.

bootstrap.sh currently assumes imagefactory is already installed (though not necessarily on loacalhost). The user should have the option of building/installing it locally.

One of three conductor processes to manage as part of:
#20

Starting off with a disclaimer: this is a small bug fix that may not be worth merging into master, especially as aeolus-image-rubygem will be going away with the arrival Tim and imgfac2. Therefore, I have not created a pull request for it yet.

From
cwolferh@e6ac8237769a,
"The puppet code was hardcoded to look for a locally built 0.6.0
version of the aeolus-image-rubygem which no longer gets built (the
current version is 0.7.0) in a typical invocation of bootstrap.sh.
However, in practice this was not breaking anything in conductor since
aeolus-image-rubygem was bundle-installed as typical rubygem.

What would break is if a user requested a specific version of
aeolus-image-rubygem by setting FACTER_AEOLUS_IMAGE_RUBYGEM_BRANCH,
for example."

Dev-tools supports rbenv while Conductor uses rvm. Both tools have the same purpose and I think we should use only one of them in all Aeolus subprojects.

bootstrap.sh uses sqlite. For QE testing purposes, we would need postgres db.
Running bootstrap and then executing tests reusts errors in the log about the db being locked.

One of the three conductor processes (part of #20 ).

platform rhel6.2
steps to reproduce:

export RBENV_VERSION=1.9.3-p362; curl https://raw.github.com/aeolus-incubator/dev-tools/master/bootstrap.sh | /bin/bash -x

then initialize rbenv

go to conductor/src

run bundle exec rails c

get:
[root@hp-magnycours-02 src]# bundle exec rails c
===== boot.rb will try and use bundler =========
/root/.rbenv/versions/1.9.3-p362/lib/ruby/1.9.1/irb/completion.rb:9:in require': cannot load such file -- readline (LoadError) from /root/.rbenv/versions/1.9.3-p362/lib/ruby/1.9.1/irb/completion.rb:9:in<top (required)>'

Conductor will soon use Tim by default; dev-tools needs to be ready for the switch. For now, that probably means creating a tim-stable branch within dev-tools that checks out Conductor's tim-stable branch.

error message in log: Using randomly generated secret token: No such file or directory - /etc/aeolus-conductor/secret_token

easy to fix, will try to do it asap

When this lands in conductor:
rails/rails#8548

we can undo:
#44

Similar to #26 , only this will be the first post-tim release.

Here is what the proposed release would look like: https://github.com/cwolferh/dev-tools/compare/2013-jan-1-release

It seems to work well in my testing except for a minor bug addressed by #27 . Therefore, we will want DEV_TOOLS_BRANCH updated to reflect a newer commit that includes this pull request (once it is merged) before the release is ready to go.

This should allow for specification of postgresql username and password via environment variables, as well as configuration of the database name conductor will be using.

We may want to create a Releases github wiki page for this project with links to each release. The first release is pre-tim aka 2012-Dec-1.

I have an initial pass at the release: https://github.com/cwolferh/dev-tools/blob/2012-dec-1-release/releases/2012-dec-1.sh . After some initial testing, it looks like it is working against the classic imagewarehouse / imagefactory setup. Note that this "release script" requires the user to edit 3 environment variables related to imagewarehouse / imagefactory. If this script looks good to other folks and we want to have releases in the dev-tools/release dir, I'll submit a pull request. Note that the release script doesn't really have to live in dev-tools as it is responsible for checking out the relevant branch or tag from dev-tools itself, it just seems like the most convenient place to put it.

Basically need a reliable/easy way to restart the conductor process started by bootstrap. Either adding a rake task, script, or service would be better than requiring spawning/disown.ing individual process which was problematic at one point.