vagrant about cucumber-chef HOT 27 CLOSED

I think he touches on this is the book, but it was mainly a speed decision. Since building a reprovisioning happens for each test, boot time was important, and LXC was the best, so he went with that :)

I'm sure there'd be no objection is someone coded it up, but it wasn't a best first focus, I'm guessing

from cucumber-chef.

Just happened to be watching this video last night:
http://skillsmatter.com/podcast/agile-scrum/cucumber-chef

[00:17:30] Talks about Vagrant as a possible approach before settling on LXC
[00:31:00] Started with Vagrant support. Code still there for it somewhere.

from cucumber-chef.

Since this is using fog it would seem fairly easy to use the fog VirtualBox provider. Since vagrant uses VirtualBox I can't see why you would really need vagrant in the picture; using LXC via VirtualBox should get the job done and saves you having to spin VMs up and down.

from cucumber-chef.

that would be ideal, this would avoid having ec2s being left running especially when you have several of your employees using it.

from cucumber-chef.

This is the next feature to be implemented.

from cucumber-chef.

any progress on this piece?

from cucumber-chef.

Sorry for the late response; as of yet no. This is definitely one of the next high priority features.

from cucumber-chef.

Thanks!

-- Jay

On Jul 19, 2012, at 9:34 PM, Zachary [email protected] wrote:

Sorry for the late response; as of yet no. This is definitely one of the next high priority features.

---

Reply to this email directly or view it on GitHub:
#46 (comment)

from cucumber-chef.

Vagrant / VBox support in Test-Kitchen has been open sourced, and gives a good approach which we can consider adopting. Additionally, the new machine independent architecture of Vagrant opens the possibilities of developing KVM or LXC based local solutions. This is being explored.

from cucumber-chef.

Hi any news? Why we do not just use fog to deal with EC2 or Vagrant?

from cucumber-chef.

@chinadialogue Cucumber-Chef already uses Fog for EC2, which is quite clear from the source code if one looked at it; we will be supporting things like VirtualBox/Vagrant soon, unfortunately I can't spend 40 hour a week coding on this project as much as I'd love to; have to pay the bills and support the family and all that stuff. We understand the need for VirtualBox/Vagrant support. Expansion to support other virtualization mediums is now #1 on my list. If I'm going too slow you can always submit a pull request! :)

from cucumber-chef.

@chinadialogue it would be really helpful to understand what the use case is. What specifically about the current model doesn't work for you? If Vagrant support were added, what would it need to do to be useful? And what benefit would you get out of it?

from cucumber-chef.

The number one reason for wanting Vagrant support, at least for me, is the expense of running an AWS instance.

from cucumber-chef.

++ on the expense.

The other one for me would having the ability to work offline. If I am dependent on aws I need an internet connection thus cutting myself out if I'm offline.

from cucumber-chef.

@pikesley 👍
@jperry 👍

from cucumber-chef.

This is a very high priority item/requirement for myself. We are using Cucumber-Chef at my "day job" and even thou we have the money to spin up EC2 instances for these purposes there is a large desire to have it done via local virtualization. I alone can see a massive benefit for this; no EC2 costs, ability to use SSD's (the time savings here alone would be huge) and as @jperry mentioned you could work offline.

I should mention the reason I haven't made this happen yet is because I've been sidetracked for the past few months at work on our production systems and my weekends, when I have free time, have been spent keeping my personal internet assets afloat.

from cucumber-chef.

👍

from cucumber-chef.

vagrant support is in master and the latest rc releases now.

from cucumber-chef.

Hello! "Vagrant guy" here.

This is fantastic news. I think this is a good idea now because I think its going to be a great idea a few months from now. I'll explain in a moment. I also took a look at your implementation and want to discuss it a bit.

Why this is a Stupidly Awesome Fantastic Idea

You may or may not know, but Vagrant 1.1 is nearing completion. Vagrant 1.1 is the first release to fully separate VirtualBox from Vagrant, meaning Vagrant can power any kind of machine, whether that be on local virtualization, remote cloud, LXC containers, whatever. As an example, you can see I already have a beta of a VMware Fusion provider done.

Why is this awesome for YOU? Well, it takes the pain of managing and launching infrastructure away from projects like cucumber-chef. Imagine that Vagrant is your only layer to getting test infrastructure. Cucumber-chef can rely on that for... AWS machines, LXC containers, Fusion machines, etc. This allows you to focus on what you probably care about most: the interface and test functionality, rather than infrastructure management headaches. Hurrah!

Implementation Notes

Now let's talk about your implementation (as of the time of this comment). I only need to link to one line:

You are using the Vagrant internals directly. This is absolutely correct for Vagrant 1.0.x, but is unfortunately not future-proof for 1.x and onwards. Vagrant 1.1+ will no longer be distributed as a RubyGem. It will be installers only. There are many reasons for this and I plan on writing about them shortly, but just understand that that is the state of things.

The main issue with this is that you can't just require "vagrant" and use the internals anymore. Vagrant runs its own isolated Ruby. PLUS, you don't want to, because if Vagrant changes then you get screwed.

Therefore, the recommended path for projects like yours is to actually use subprocesses and shell out. Run an actual vagrant up, run a vagrant status, etc. Every command in Vagrant will (but does not yet) have a --machine-readable flag soon which will make the output consumable by programs like yours in an easy way and will contain far more information about what is going on than normal human output. Neat!

If you have any questions, I'm now watching this thread, so feel free to ask. Thanks so much!

from cucumber-chef.

@mitchellh Awesome, thanks for the information! I've actually been toying with a similar idea for how I interface with knife. I was originally shelling out for the chef related bits; then switched over to directly calling the chef code. This of course has bitten me when methods have changed internally between chef versions.

Knowing this I have been feeling more and more inclined to switch back over to shelling out for knife bits, and now even more so since this will be Vagrants new pattern.

I'm very eager to hear more about the changes coming to Vagrant! Please keep me posted!

from cucumber-chef.

I've switched over to running vagrant as suggested; except for the status information. I will not be able to pull that off or the rest of the integration until I have more details on what I'll need to parse, etc (re: --machine-readable flag) and a pre-release version of vagrant to play with.

But at least the ball is rolling.

I've also done similar with the knife logic; I'll save that for another post.

from cucumber-chef.

I'd love some feedback on the 2.1.0 RC's, mainly the latest one re: Vagrant. Issues, Comments, Death Threats?

from cucumber-chef.

Maybe I'm being a clueless prawn (hey, never bet against it) but I've just installed 2.1.0.rc.4 and I can't see where or how to configure Vagrant. cucumber-chef init does the familiar "Tell me your AWS creds" thing, and the generated config.rb still says "Vagrant: TODO"

I guess I'm missing something. Can somebody point me in the right direction please?

from cucumber-chef.

Sorry; Ya I'm saving updating the UI-related bits last. I posted another "issue" with the new config options. You'll need to bring the generated config file in line with this. For now you can just skip the AWS questions (answer junk if you want).

While the config file itself is Ruby; the config options themselves are displayed in YAML to the end user. You can use the command cucumber-chef displayconfig.

New Config Options: #92

from cucumber-chef.

I found a few issues today that I've fixed. I'm going to push out another rc tonight. You'll likely have some issues with the chef client runs in the interim; but the test labs should build OK.

And next logical step will be creating "test lab" base boxes we can ship out so people can avoid that whole process. All in all it takes me about 15 minutes to bootstrap a Vagrant test lab as is (VM w/8 cores & 8GB of RAM; don't ask what my system specs are lol). Then another 10 minutes or so for LXC to build it's base image. So right now about 30 minutes to create the entire lab from scratch. Once LXC has it's base image in place; it only takes about 30 seconds to create an LXC from scratch. That is with no SSD's as well. Put SSD's in the mix and I'll bet you can create/destroy many containers per minute meaning highly feasible testing of your entire chef-repo.

I was stuck using AWS test labs at work because my Mac at the office (8 cores, 8GB of RAM) can't seem to run a VM with 2 cores and 2GB of RAM worth a foo.

$ bin/cucumber-chef setup
cucumber-chef v2.1.0.rc.4
Creating VAGRANT instance completed in 38.8245 seconds.
Bootstrapping VAGRANT instance completed in 517.1095 seconds.
Waiting for the chef-server completed in 0.1038 seconds.
Waiting for the chef-server-webui completed in 0.1006 seconds.
Downloading chef-server credentials completed in 0.7058 seconds.
Building 'cc-knife' configuration completed in 0.2009 seconds.
Uploading 'cucumber-chef' cookbooks completed in 6.2517 seconds.
Uploading 'cucumber-chef' roles completed in 1.7134 seconds.
Tagging 'cucumber-chef' node completed in 1.7135 seconds.
Setting 'cucumber-chef' run list completed in 1.9133 seconds.
Performing chef-client run completed in 260.9778 seconds.
Downloading proxy SSH credentials completed in 0.3011 seconds.
Rebooting the test lab completed in 21.2678 seconds.
Waiting for the chef-server completed in 0.1004 seconds.
Waiting for the chef-server-webui completed in 0.1005 seconds.

If you are using AWS, be sure to log into the chef-server webui and change the default admin password at least.

Your test lab has now been provisioned!  Enjoy!

+-------------------------------------------------------------------+
|                      PROVIDER: Cucumber::Chef::Provider::Vagrant  |
|                            ID: default                            |
|                         STATE: not_created                        |
|                      USERNAME: vagrant                            |
|                    IP ADDRESS: 127.0.0.1                          |
|                          PORT: 2222                               |
|               CHEF-SERVER API: http://127.0.0.1:4000              |
|             CHEF-SERVER WEBUI: http://127.0.0.1:4040              |
|      CHEF-SERVER DEFAULT USER: admin                              |
|  CHEF-SERVER DEFAULT PASSWORD: p@ssw0rd1                          |
+-------------------------------------------------------------------+

I must query Vagrant's state too soon because it always seems to say "not_created". Another status command and we see it is really up and alive:

$ bin/cucumber-chef status
cucumber-chef v2.1.0.rc.4
+-------------------------------------------------------------------+
|                      PROVIDER: Cucumber::Chef::Provider::Vagrant  |
|                            ID: default                            |
|                         STATE: running                            |
|                      USERNAME: vagrant                            |
|                    IP ADDRESS: 127.0.0.1                          |
|                          PORT: 2222                               |
|               CHEF-SERVER API: http://127.0.0.1:4000              |
|             CHEF-SERVER WEBUI: http://127.0.0.1:4040              |
|      CHEF-SERVER DEFAULT USER: admin                              |
|  CHEF-SERVER DEFAULT PASSWORD: p@ssw0rd1                          |
+-------------------------------------------------------------------+

$ bin/cucumber --tags @timezone
Using the default profile...
Code:
  * features/support/env.rb
  * features/support/cc-hooks.rb
  * cucumber-chef v2.1.0.rc.4

Features:
  * features/timezone/timezone.feature
Parsing feature files took 0m0.081s

@timezone
Feature: Perform test driven infrastructure with Cucumber-Chef
  In order to learn how to develop test driven infrastructure
  As an infrastructure developer
  I want to better understand how to use Cucumber-Chef

  Background:                                                         # features/timezone/timezone.feature:7
    * I have a server called "timezone"                               # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:22
    * "timezone" is running "ubuntu" "lucid"                          # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:26
    * "timezone" should be persistent                                 # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:34
    * "timezone" has been provisioned                                 # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:46
    * the following cookbooks have been uploaded:                     # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/chef_steps.rb:57
      | cookbook | cookbook_path |
      | timezone | ./cookbooks   |
    * the "timezone" recipe has been added to the "timezone" run list # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:54
    * the chef-client has been run on "timezone"                      # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:62
    * I ssh to "timezone" with the following credentials:             # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/ssh_steps.rb:28
      | username | keyfile |
      | $lxc$    | $lxc$   |

  Scenario: System timezone is set and defaults to UTC.               # features/timezone/timezone.feature:21
    And I run "cat /etc/timezone"                                     # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/ssh_steps.rb:62
    Then I should see "UTC" in the output                             # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/ssh_steps.rb:68

1 scenario (1 passed)
10 steps (10 passed)
9m11.348s

Running the same test again; now that the LXC is up and running take 12s:

$ bin/cucumber --tags @timezone
Using the default profile...
Code:
  * features/support/env.rb
  * features/support/cc-hooks.rb
  * cucumber-chef v2.1.0.rc.4

Features:
  * features/timezone/timezone.feature
Parsing feature files took 0m0.084s

@timezone
Feature: Perform test driven infrastructure with Cucumber-Chef
  In order to learn how to develop test driven infrastructure
  As an infrastructure developer
  I want to better understand how to use Cucumber-Chef

  Background:                                                         # features/timezone/timezone.feature:7
    * I have a server called "timezone"                               # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:22
    * "timezone" is running "ubuntu" "lucid"                          # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:26
    * "timezone" should be persistent                                 # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:34
    * "timezone" has been provisioned                                 # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:46
    * the following cookbooks have been uploaded:                     # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/chef_steps.rb:57
      | cookbook | cookbook_path |
      | timezone | ./cookbooks   |
    * the "timezone" recipe has been added to the "timezone" run list # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:54
    * the chef-client has been run on "timezone"                      # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/provision_steps.rb:62
    * I ssh to "timezone" with the following credentials:             # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/ssh_steps.rb:28
      | username | keyfile |
      | $lxc$    | $lxc$   |

  Scenario: System timezone is set and defaults to UTC.               # features/timezone/timezone.feature:21
    And I run "cat /etc/timezone"                                     # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/ssh_steps.rb:62
    Then I should see "UTC" in the output                             # vendor/checkouts/cucumber-chef/lib/cucumber/chef/steps/ssh_steps.rb:68

1 scenario (1 passed)
10 steps (10 passed)
0m12.135s

from cucumber-chef.

IMPORTANT

For Vagrant; here is my config.rb and what to add to Vagrantfile (like I said it's beta; I'll have the loose ends tied up soon enough)

config.rb:
#92 (comment)

Vagrantfile:
#92 (comment)

from cucumber-chef.

Closing this out since vagrant support has been in for a while now.

from cucumber-chef.