chef-partners / azure-chef-extension Goto Github PK

The development repository for the Chef Extension available through Azure

License: Apache License 2.0

Nix 0.11% Ruby 68.93% PowerShell 20.12% Shell 6.39% HTML 2.27% Batchfile 0.73% Python 0.11% Makefile 1.33%

azure-chef-extension's Issues

autoUpdateClient=false being ignored

This happened last year, I just found this project today. This might be out of date, but thought its worth posting in case;

The Azure Plugin for Chef uninstalled our ChefClient Agent and left all of our FrontEnd VMs without the ChefClient installed.

It appears Azure attempted to upgrade the ChefClient -- even though we have autoUpdateClient set to false. Our Chef client plugin was 1210.12.4.1 -- on 20th Aug it appears it tried to upgrade to 1210.12.4.1000.

I can see in the logs that the uninstall did not respect autoUpdateClient = false and uninstalled the ChefClient.

It then executed install.cmd for 1210.12.4.1000 which did respect autoUpdateClient = false -- meaning that it did not install the ChefClient.

I have manually updated autoUpdateClient = true and ran the install.cmd and enable.cmd manually which appears to have fixed the problem.

Running as Scheduled Task instead of service

Chef recommends running the Chef Client as a Scheduled Task in Windows, rather than a Windows service. Are there any plans for this extension to choose between task and service?

is this where it comes from?

is this where MS pulls the Chef Extension from every time I install the extension as part of an ARM template? This is just a general question, but it's something that's highly important to folks behind proxies or whatever.

On another note, what version of the Chef Client is the most recent within the extension? When I push mine, I'm getting version 12.3, which is really old, and I want at least 12.7 to use some of the better DSC capabilities. 12.9 would be ideal. What version of the extension would I need to use to do that, since it appears that you're extension version is not the same as the client version?

Existing System Ruby Breaks azure-chef-extension install.sh/enable.sh

--- Versions:

OS: RHEL7
azure-chef-extension: latest

--- Problem:

As a customer, I want to be able to have a system ruby installed alongside an embedded chef-client ruby.
Redhat Satellite and other things depend on the system ruby.

When a system ruby is installed, the current version of azure-chef-extension fails while being installed.

It has the following output while manually running enable.sh directly on the system as a test, because the gem gets installed in the system ruby gem location, rather than the embedded chef-client gem location.

root@flapvm-azpcc01 Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000]# ./enable.sh

/var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000

/opt/chef/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- chef/azure/commands/enable (LoadError)

from /opt/chef/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'

from /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000/bin/chef-enable.rb:4:in `<main>'

This line is actually where the failure is occurring

azure-chef-extension/ChefExtensionHandler/bin/chef-enable.rb

Line 4 in 9e9d7b8

require 'chef/azure/commands/enable'

--- Possible fix (There is likely a better way to fix this, please do):

Add full path /opt/chef/embedded/bin to have azure-chef-extension gem installed in the
embedded chef-client gems in chef-install.sh

azure-chef-extension/ChefExtensionHandler/bin/chef-install.sh

Line 18 in 9e9d7b8

gem install "$1" --no-ri --no-rdoc
Add full path /opt/chef/embedded/bin to chef-client embedded ruby in chef-enable.sh

azure-chef-extension/ChefExtensionHandler/enable.sh

Line 17 in 9e9d7b8

ruby $CHEF_EXT_DIR/bin/chef-enable.rb

When these two changes are made, even with a system ruby present, we expect

That the azure-chef-extension gem will be installed in the correct embedded gem location
The correct embedded ruby will be called, so that the require in bin/chef-enable.rb will not fail

Almost all of my extensions are failing after the release of 1210.12.104.1000

New-AzureRmResourceGroupDeployment : 11:27:09 AM - Handler 'Chef.Bootstrap.WindowsAzure.ChefClient' has reported failure for VM Extension 'ChefExtension' with terminal error code '1007' and error
message: 'Install failed for plugin (name: Chef.Bootstrap.WindowsAzure.ChefClient, version 1210.12.104.1000) with exception Command
C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.104.1000\install.cmd of Chef.Bootstrap.WindowsAzure.ChefClient has not exited on time! Killing it...'

Even when I was on 103, I would get this error about 10% of the time but it appears to be almost every bootstrap now.

Not working with CentOS VMs?

I've tried creating both CentOS 6.6 and 7.0 VMs through the preview portal and the bootstrapping process fails.

CommandExecution.log

2015/03/30 12:16:53 Found RuntimeSettings for Chef.Bootstrap.WindowsAzure.LinuxChefClient V 11.16.4.2
2015/03/30 12:16:53 Spawned install.sh PID 1759
2015/03/30 12:16:58 Process 1759 returned non-zero exit code (1)
2015/03/30 12:16:58 Installation failed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2

Manually running install.sh

/var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2
Installing Chef
installing with rpm.../var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2/bin/..//installer/chef-client-latest.rpm
error: open of /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2/bin/..//installer/chef-client-latest.rpm failed: No such file or directory
Chef Client installation failed

The installer directory only contains chef-client-latest.deb.

ARM Chef Extension issue JSON Escaping Keys not working

I am trying to provision the Linux Chef Extension ARM template using ARM and I am unsuccessful.
It worked with the classic portal. The issue seems to be with the validation key parameter in the ARM template
The classic portal expects a file name, whereas the ARM template expects a string.
I have tried the template parameter with both validation key filepath and RSA key as well but the chef-client is unable to register. The chef-client fails to register to chef-server, and i believe it's because it can't parse JSON escape keys. I tried the json parameter file and noticed that the validation key should have the entire contents of the .pem file with \n added. I used this json parameter and my .pem contents and provisioned the VM and chef extension failed to register.

Process I followed is :

Launch the my template: https://github.com/srakesh28/azure-iaasv2-arm/blob/master/step2-linuxvm/azuredeploy-chefextention.json
Provides all the chef parameters in the the portal. Here are sample values
nodename :
Chef Server url :
Validation client name -
Validation Key
Json escape the contents of orgname-validator.pem
Run the template.
Server are deployed Chef Client is installed and I can see in Portal but Client does not register to Chef-server.

Thank you

Knife azure: bootstrap-protocol cloudapi fails with 400 if you specify a runlist

See this log below -- the Azure extension fails when a runlist is specified -- this is due to failure to escape the runlist provided by some command line tools, including knife-azure. This regression was introduced in extension version https://github.com/chef-partners/azure-chef-extension/tree/1206.12.3.0.

# Logfile created on 2015-05-12 14:26:35 +0000 by logger.rb/41954
[2015-05-12T14:27:55+00:00] INFO: *** Chef 12.3.0 ***
[2015-05-12T14:27:55+00:00] INFO: Chef-client pid: 2992
[2015-05-12T14:27:59+00:00] INFO: Chef Client Service initialized
[2015-05-12T14:28:04+00:00] INFO: Next chef-client run will happen in 0.0015101726050843478 seconds
[2015-05-12T14:28:06+00:00] INFO: Reconfiguring with startup parameters
[2015-05-12T14:28:10+00:00] INFO: Chef-Client service is starting a chef-client run...
[2015-05-12T14:28:11+00:00] INFO: Starting chef-client in a new process
[2015-05-12T14:28:14+00:00] WARN: Chef client 2992 is running, will wait for it to finish and then run.
[2015-05-12T14:28:58+00:00] INFO: Client key c:/chef/client.pem is not present - registering
[2015-05-12T14:29:02+00:00] INFO: HTTP Request Returned 404 Object Not Found: error
[2015-05-12T14:29:03+00:00] INFO: Setting the run_list to ["\"devbox\""] from CLI options
[2015-05-12T14:29:03+00:00] INFO: Run List is [recipe["devbox"]]
[2015-05-12T14:29:03+00:00] INFO: Run List expands to ["devbox"]
[2015-05-12T14:29:03+00:00] INFO: Starting Chef Run for az-86984886c213.az-86984886c213.d4.internal.cloudapp.net
[2015-05-12T14:29:03+00:00] INFO: Running start handlers
[2015-05-12T14:29:03+00:00] INFO: Loading Handler environment...
[2015-05-12T14:29:03+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0
[2015-05-12T14:29:03+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status/0.status
[2015-05-12T14:29:03+00:00] INFO: Reporting heartbeat...
[2015-05-12T14:29:03+00:00] INFO: Start handlers complete.
[2015-05-12T14:29:04+00:00] INFO: HTTP Request Returned 400 Bad Request: error
[2015-05-12T14:29:04+00:00] ERROR: Running exception handlers
[2015-05-12T14:29:04+00:00] INFO: Loading Handler environment...
[2015-05-12T14:29:04+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0
[2015-05-12T14:29:04+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status/0.status
[2015-05-12T14:29:04+00:00] INFO: Reporting heartbeat...
[2015-05-12T14:29:04+00:00] ERROR: Exception handlers complete
[2015-05-12T14:29:04+00:00] FATAL: Stacktrace dumped to c:/chef/cache/chef-stacktrace.out
[2015-05-12T14:29:04+00:00] INFO: Sending resource update report (run-id: 08086e7d-4acd-48eb-a64a-a96ab7848a1d)
[2015-05-12T14:29:06+00:00] FATAL: Net::HTTPServerException: 400 "Bad Request"
[2015-05-12T14:29:15+00:00] INFO: *** Chef 12.3.0 ***
[2015-05-12T14:29:15+00:00] INFO: Chef-client pid: 2720
[2015-05-12T14:29:46+00:00] INFO: Run List is []
[2015-05-12T14:29:46+00:00] INFO: Run List expands to []
[2015-05-12T14:29:46+00:00] INFO: Starting Chef Run for az-86984886c213.az-86984886c213.d4.internal.cloudapp.net
[2015-05-12T14:29:46+00:00] INFO: Running start handlers
[2015-05-12T14:29:46+00:00] INFO: Loading Handler environment...
[2015-05-12T14:29:46+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0
[2015-05-12T14:29:46+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status/0.status
[2015-05-12T14:29:46+00:00] INFO: Reporting heartbeat...
[2015-05-12T14:29:46+00:00] INFO: Start handlers complete.
[2015-05-12T14:29:47+00:00] INFO: Loading cookbooks []
[2015-05-12T14:29:47+00:00] WARN: Node az-86984886c213.az-86984886c213.d4.internal.cloudapp.net has an empty run list.
[2015-05-12T14:29:48+00:00] INFO: Chef Run complete in 2.002575 seconds
[2015-05-12T14:29:48+00:00] INFO: Running report handlers
[2015-05-12T14:29:48+00:00] INFO: Loading Handler environment...
[2015-05-12T14:29:48+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0
[2015-05-12T14:29:48+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status/0.status
[2015-05-12T14:29:48+00:00] INFO: Reporting heartbeat...
[2015-05-12T14:29:48+00:00] INFO: Report handlers complete
[2015-05-12T14:29:48+00:00] INFO: Sending resource update report (run-id: 36ca6796-f1c4-4ceb-be3d-450eb0917335)
[2015-05-12T14:29:55+00:00] INFO: Child process exited (pid: 1808)
[2015-05-12T14:29:55+00:00] INFO: Next chef-client run will happen in 1800.918139971232 seconds
[2015-05-12T14:59:57+00:00] INFO: Reconfiguring with startup parameters
[2015-05-12T15:00:01+00:00] INFO: Chef-Client service is starting a chef-client run...
[2015-05-12T15:00:02+00:00] INFO: Starting chef-client in a new process
[2015-05-12T15:00:15+00:00] INFO: *** Chef 12.3.0 ***
[2015-05-12T15:00:15+00:00] INFO: Chef-client pid: 904
[2015-05-12T15:00:48+00:00] INFO: Run List is []
[2015-05-12T15:00:48+00:00] INFO: Run List expands to []
[2015-05-12T15:00:48+00:00] INFO: Starting Chef Run for az-86984886c213.az-86984886c213.d4.internal.cloudapp.net
[2015-05-12T15:00:48+00:00] INFO: Running start handlers
[2015-05-12T15:00:48+00:00] INFO: Loading Handler environment...
[2015-05-12T15:00:48+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0
[2015-05-12T15:00:48+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status/0.status
[2015-05-12T15:00:48+00:00] INFO: Reporting heartbeat...
[2015-05-12T15:00:48+00:00] INFO: Start handlers complete.
[2015-05-12T15:00:50+00:00] INFO: Loading cookbooks []
[2015-05-12T15:00:50+00:00] WARN: Node az-86984886c213.az-86984886c213.d4.internal.cloudapp.net has an empty run list.
[2015-05-12T15:00:51+00:00] INFO: Chef Run complete in 3.801098 seconds
[2015-05-12T15:00:51+00:00] INFO: Running report handlers
[2015-05-12T15:00:51+00:00] INFO: Loading Handler environment...
[2015-05-12T15:00:51+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0
[2015-05-12T15:00:51+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status/0.status
[2015-05-12T15:00:51+00:00] INFO: Reporting heartbeat...
[2015-05-12T15:00:51+00:00] INFO: Report handlers complete
[2015-05-12T15:00:51+00:00] INFO: Sending resource update report (run-id: abb99b2a-6668-414d-8468-52b521ffd351)
[2015-05-12T15:00:57+00:00] INFO: Child process exited (pid: 1808)
[2015-05-12T15:00:58+00:00] INFO: Next chef-client run will happen in 1800.3571128660233 seconds
[2015-05-12T15:30:59+00:00] INFO: Reconfiguring with startup parameters
[2015-05-12T15:31:03+00:00] INFO: Chef-Client service is starting a chef-client run...
[2015-05-12T15:31:04+00:00] INFO: Starting chef-client in a new process
[2015-05-12T15:31:17+00:00] INFO: *** Chef 12.3.0 ***
[2015-05-12T15:31:17+00:00] INFO: Chef-client pid: 2304
[2015-05-12T15:31:49+00:00] INFO: Run List is []
[2015-05-12T15:31:49+00:00] INFO: Run List expands to []
[2015-05-12T15:31:49+00:00] INFO: Starting Chef Run for az-86984886c213.az-86984886c213.d4.internal.cloudapp.net
[2015-05-12T15:31:49+00:00] INFO: Running start handlers
[2015-05-12T15:31:49+00:00] INFO: Loading Handler environment...
[2015-05-12T15:31:49+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0
[2015-05-12T15:31:49+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status/0.status
[2015-05-12T15:31:49+00:00] INFO: Reporting heartbeat...
[2015-05-12T15:31:49+00:00] INFO: Start handlers complete.
[2015-05-12T15:31:50+00:00] INFO: Loading cookbooks []
[2015-05-12T15:31:50+00:00] WARN: Node az-86984886c213.az-86984886c213.d4.internal.cloudapp.net has an empty run list.
[2015-05-12T15:31:51+00:00] INFO: Chef Run complete in 2.023516 seconds
[2015-05-12T15:31:51+00:00] INFO: Running report handlers
[2015-05-12T15:31:51+00:00] INFO: Loading Handler environment...
[2015-05-12T15:31:51+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0
[2015-05-12T15:31:51+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1206.12.3.0\Status/0.status
[2015-05-12T15:31:51+00:00] INFO: Reporting heartbeat...
[2015-05-12T15:31:51+00:00] INFO: Report handlers complete
[2015-05-12T15:31:51+00:00] INFO: Sending resource update report (run-id: 3f09e873-614d-4882-89bc-5fd4d630ef13)
[2015-05-12T15:31:58+00:00] INFO: Child process exited (pid: 1808)
[2015-05-12T15:31:59+00:00] INFO: Next chef-client run will happen in 1800.153187507208 seconds

Type handler version '1210.12' could not be found in the extension repository

This issue seems to have reappeared suddenly today. There is an earlier (July-August) issue that was closed when it was resolved. We have tried US West and North Europe data centers and it is the same in both.

This is breaking a critical demo application for one of our customers so we are requesting attention on this ASAP.

Azure Server Create fail with "wagent provisioning" state not reached after 5 minutes

We are currently trying out the Knife Azure plugin, and using the ASM mode Server Create command.
It's constantly generating an error similar to the following:

..................ERROR: Resource extension state 'wagent provisioning' not reached after 5 minutes. chef-service enabled
ERROR: Failed to create the server -- exception being rescued: Verify connectivity to Azure and subscription resource limit compliance (e.g. maximum CPU core limits) and try again.
WARNING: Cleaning up resources...

Here are some relevant details:

we are using the azure-chef extension version 1210.12.102.1000, set via following
knife[:azure_chef_extension_version] = '1210.12'
We are trying to provision a "Large" vm with "Windows 2012 R2"
We are using a hosted chef account for provisioning a simple webserver cookbook
After the Azure Knife exception (described above), the VM is still in Running state, and accessible via RDP.
On further investigation it appears that the Chef Extension is installed on the VM and logs are available under C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient
C:\chef directory also appears to have all the files for the cookbooks downloaded and available.

I have also attached the log from azure chef extension plugin. Any immediate help would be well appreciated.

chef-client.log.txt
CommandExecution.log.txt
CommandExecution_0.log.txt

Please add Azure KeyVault support

I would like to store my pems in Azure KeyVault and reference them from an ARM template.

Extension does not create client.rb, install fails

Using the chef-json-parameters-linux-vm quickstart template for ARM deployments, and chef fails to start, complaining that there is no /etc/chef/client.rb file. When I log into the server, I only see a file called 0.settings with all my parameters in the /etc/chef/ directory and nothing else.

Here's the CommandExecutionLog:
2016/04/20 23:05:44 Downloading plugin manifest: Chef.Bootstrap.WindowsAzure.LinuxChefClient from http://ardfepirv2sn3prdstr05.blob.core.windows.net/0fa68ca509a643f49117169ef78baf70/Chef.Bootstrap.WindowsAzure_LinuxChefClient_ussouth_manifest.xml
2016/04/20 23:05:44 Plugin server is: ardfepirv2sn3prdstr05.blob.core.windows.net
2016/04/20 23:05:45 Bundle URI = http://ardfepirv2sn3prdstr05.blob.core.windows.net/0fa68ca509a643f49117169ef78baf70/Chef.Bootstrap.WindowsAzure__LinuxChefClient__1210.12.102.1000
2016/04/20 23:05:45 Plugin bundlehttp://ardfepirv2sn3prdstr05.blob.core.windows.net/0fa68ca509a643f49117169ef78baf70/Chef.Bootstrap.WindowsAzure__LinuxChefClient__1210.12.102.1000downloaded successfully length = 70669
2016/04/20 23:05:45 Extracted http://ardfepirv2sn3prdstr05.blob.core.windows.net/0fa68ca509a643f49117169ef78baf70/Chef.Bootstrap.WindowsAzure__LinuxChefClient__1210.12.102.1000 to /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000
2016/04/20 23:05:45 Found RuntimeSettings for Chef.Bootstrap.WindowsAzure.LinuxChefClient V 1210.12.102.1000
2016/04/20 23:05:45 Spawned install.sh PID 1788
2016/04/20 23:06:55 installCommand completed.
2016/04/20 23:06:55 Installation completed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000
2016/04/20 23:06:55 Found RuntimeSettings for Chef.Bootstrap.WindowsAzure.LinuxChefClient V 1210.12.102.1000
2016/04/20 23:06:55 Spawned enable.sh PID 1879
2016/04/20 23:07:00 Process 1879 returned non-zero exit code (1)
2016/04/20 23:07:00 Enable failed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000
2016/04/20 23:07:00 Processing completed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000
2016/04/20 23:07:00 Finished processing ExtensionsConfig.xml

And the custom.log:
/var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000
Call for Checking linux distributor
Starting installation process:
23:05:45
Checking for curl...
Detected curl...
Reading chef-client version from settings file
Installing chef-client package
Selecting previously unselected package chef.
(Reading database ... 52410 files and directories currently installed.)
Preparing to unpack /tmp/tmp.CtiseXIt07/chef ...
Unpacking chef (12.9.38-1) ...
Setting up chef (12.9.38-1) ...
Thank you for installing Chef!
[Wed Apr 20 23:06:48 UTC 2016] Package Chef installed successfully.
End of installation:
23:06:48
[Wed Apr 20 23:06:49 UTC 2016] Installing Azure Chef Extension gem
Successfully installed azure-chef-extension-0.0.1
1 gem installed
[Wed Apr 20 23:06:51 UTC 2016] Azure Chef Extension gem installation succeeded

Deployment output:
New-AzureRmResourceGroupDeployment : 16:23:02 - Resource Microsoft.Compute/virtualMachines/extensions
'akcheflin6/LinuxChefExtension' failed with message 'The resource operation completed with terminal provisioning state 'Failed'.'
At C:********\linux_vm_deploy_script_LOCAL.ps1:54 char:1

New-AzureRmResourceGroupDeployment -Name $deployName -ResourceGroupNa ...

- CategoryInfo          : NotSpecified: (:) [New-AzureRmResourceGroupDeployment], Exception
- FullyQualifiedErrorId : Microsoft.Azure.Commands.Resources.NewAzureResourceGroupDeploymentCommand

New-AzureRmResourceGroupDeployment : 16:23:02 - VM has reported a failure when processing extension 'LinuxChefExtension'. Error
message: "chef-service enable failed - ["Error enabling chef-client service - No such file or directory @ rb_sysopen -
/etc/chef/client.rb - Check log file for details", "error"]".
At C:********\linux_vm_deploy_script_LOCAL.ps1:54 char:1

New-AzureRmResourceGroupDeployment -Name $deployName -ResourceGroupNa ...

- CategoryInfo          : NotSpecified: (:) [New-AzureRmResourceGroupDeployment], Exception
- FullyQualifiedErrorId : Microsoft.Azure.Commands.Resources.NewAzureResourceGroupDeploymentCommand

It's like the extension fails silently, and then continues installing Chef without doing anything with my parameters I pass it. Help!

Chef Extension Fails frequently with the following error

{ "status": "Failed", "error": { "code": "ResourceDeploymentFailure", "message": "The resource operation completed with terminal provisioning state 'Failed'.", "details": [ { "code": "VMExtensionHandlerNonTransientError", "message": "Handler 'Chef.Bootstrap.WindowsAzure.ChefClient' has reported failure for VM Extension 'chefExtension' with terminal error code '1007' and error message: 'Install failed for plugin (name: Chef.Bootstrap.WindowsAzure.ChefClient, version 1210.12.101.1000) with exception Command C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.101.1000\install.cmd of Chef.Bootstrap.WindowsAzure.ChefClient has exited with Exit code: 1'" } ] } }

Configuration error LoadError: cannot load such file -- chef/azure/chefhandlers/start_handler

I am running several Ubuntu 14.04 Machines on azure with this extension.

We added the extension to the machines though the azure portal about a year ago. It worked but has always given back the status of installing in the azure portal.

We had no problems besides the extension showing that it was Installing in the portal until today.

Our Team city server out of no where hit the error below when trying to run chef-client on one of the nodes.

[2015-12-10T21:29:13+00:00] FATAL: Configuration error LoadError: cannot load such file -- chef/azure/chefhandlers/start_handler
[2015-12-10T21:29:13+00:00] FATAL:   /etc/chef/client.rb:15:in `from_string'
[2015-12-10T21:29:13+00:00] FATAL: Aborting due to error in '/etc/chef/client.rb'

I checked /etc/client.rb and nothing seemed off but it was hanging on the bellow lines.

#Add support to use chef Handlers for heartbeat and
#status reporting to Azure
require 'chef/azure/chefhandlers/start_handler'
require 'chef/azure/chefhandlers/report_handler'
require 'chef/azure/chefhandlers/exception_handler'

start_handlers << AzureExtension::StartHandler.new('/var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2')
report_handlers << AzureExtension::ReportHandler.new('/var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2')
exception_handlers << AzureExtension::ExceptionHandler.new('/var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2')

I had to comment all the above lines out to have chef-client run properly.

I did check to see what version of the extension was on the machine. I checked in /var/lib/waaget and found two folders in there Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.18.6.2 and Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2

Our server reported running during each chef run Starting Chef Client, version 11.16.4 I found that strange so I ran sh update.sh under /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2.

The update just installed 11.16.4.2 again. So I then ran sh uninstall.sh appeared successful.

I then changed directories to Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.18.6.2 and installed it with install.sh although it appears installed already.

I check to see if my client.rb had updated links to log files and StartHandler but it was all still pointing to /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.16.4.2. and not /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-11.18.6.2. So I changed then to the new version hoping it would fix the issue. However it did not fix the issue.

I attempted to run chef-client again but it displayed the same error as before.

[2015-12-10T21:29:13+00:00] FATAL: Configuration error LoadError: cannot load such file -- chef/azure/chefhandlers/start_handler
[2015-12-10T21:29:13+00:00] FATAL:   /etc/chef/client.rb:15:in `from_string'
[2015-12-10T21:29:13+00:00] FATAL: Aborting due to error in '/etc/chef/client.rb'

The server now reports Starting Chef Client, version 11.18.6
I even tried disable.sh and enable.sh.

Not sure where to troubleshoot next.

Seems odd that the client.rb was not updated to the new version and that the issue persisted even after uninstalling the older version and installing the newer one.

Any guidance would be greatly appreciated.

Thanks Justin P.

Installation fails behind Squid Proxy

I am struggling with this for a while:
I use this extension and it works pretty fine if I allow Outbound Internet connection with port 80 and 443. If I restrict the outbound connection where I route every connection to the internet via a Squid proxy the installation of chef-client does not work.

I added proxy configuration for the Azure VM Agent in /etc/waagent.conf and I also exported environment variables http_proxy and https_proxy. The download of the Chef Extension works, but the installation fails
2016/04/26 17:01:42 Found RuntimeSettings for Chef.Bootstrap.WindowsAzure.LinuxChefClient V 1210.12.102.1000 2016/04/26 17:01:42 Spawned install.sh PID 25047 2016/04/26 17:05:23 Process 25047 returned non-zero exit code (1) 2016/04/26 17:05:23 Installation failed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000 2016/04/26 17:05:23 Processing completed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000 2016/04/26 17:05:23 Finished processing ExtensionsConfig.xml

If I check /var/log/azure/custom.log, it shows
[Tue Apr 26 17:00:45 UTC 2016] Installing Azure Chef Extension gem [Tue Apr 26 17:00:45 UTC 2016] Azure Chef Extension gem installation failed /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000 Call for Checking linux distributor Starting installation process: 17:01:43 Checking for curl... Detected curl... Reading chef-client version from settings file Installing chef-client package [Tue Apr 26 17:05:18 UTC 2016] Unable to uninstall package Chef. End of installation: 17:05:18

If I check the network trace with tcpdump, I can see that all traffic to chef.io does not go through the proxy. What further adjustment do I need to make?

Anyway, If I try to install manually using /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000/install.sh it gives me the error gem: command not found.

if I install gem manually and execute the script again, I get the following output:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 100 56.4M 100 56.4M 0 0 23.4M 0 0:00:02 0:00:02 --:--:-- 56.4M warning: /tmp/tmp.LABP95F1eY/chef: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY

If I check /var/log/azure/custom.log again, it shows
/var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.102.1000 Call for Checking linux distributor Starting installation process: 17:40:20 Checking for curl... Detected curl... Reading chef-client version from settings file Installing chef-client package Preparing... ######################################## Updating / installing... chef-12.7.2-1.el7 ######################################## Thank you for installing Chef! [Tue Apr 26 17:40:29 UTC 2016] Package Chef installed successfully. End of installation: 17:40:29 [Tue Apr 26 17:40:29 UTC 2016] Installing Azure Chef Extension gem Successfully installed azure-chef-extension-0.0.1 1 gem installed [Tue Apr 26 17:40:31 UTC 2016] Azure Chef Extension gem installation succeeded

After the enable script enable.sh is executed, finally chef-client converges successfully.

Somehow, gem needs to be installed for installing Chef with this extension. But the chef-install.sh does not contain this part. Actually if I deploy a machine without using a proxy where I have internet connection (and where the chef extension installs chef successfully) gem will not be installed. So, how does this work? Do I really need to add a custom script extension in my ARM template where I install gem?

Looking forward to hear from you :)

Thank you!

[Security] Extension leaks sensitive information

Just tried to use the Chef Extension. The extension leaked a private key to the Azure Portal and did not provide a useful error message. We are using CentOS.

Custom json attributes in first-boot.json?

Is there a way to add custom node attributes, passed in as a json string, when bootstrapping a node? Knife bootstrap supports a "--json-attributes" flag to do this, and sets them in the first-boot.json file.

Can no longer install 12.4.1 client on Linux via ChefVMExtension

It appears that 1210.12.4.1000 LinuxChefClient was yanked and replaced with .2000 - is it to be expected that minor versions can get bumped without notice?

Scenario: I'm rebuilding my environment in an ARM template using '1210.12' as my version but get a different version of the client to the environment I created using the identical template two weeks ago. Therefore my environment is inconsistent with the testing I did two weeks ago on apparently the same version.

Also can we expect a 12.4.3 for (Windows) ChefClient soon?

thanks!

PublisherName               Type             Version
-------------               ----             -------
Chef.Bootstrap.WindowsAzure CentosChefClient 11.12.4.2
Chef.Bootstrap.WindowsAzure CentosChefClient 11.14.6.1
Chef.Bootstrap.WindowsAzure ChefClient       11.10.4
Chef.Bootstrap.WindowsAzure ChefClient       11.12.0.0
Chef.Bootstrap.WindowsAzure ChefClient       11.18.6.1
Chef.Bootstrap.WindowsAzure ChefClient       1203.12.1.1
Chef.Bootstrap.WindowsAzure ChefClient       1205.12.2.1
Chef.Bootstrap.WindowsAzure ChefClient       1207.12.3.0
Chef.Bootstrap.WindowsAzure ChefClient       1210.12.4.1000
Chef.Bootstrap.WindowsAzure LinuxChefClient  11.18.6.1
Chef.Bootstrap.WindowsAzure LinuxChefClient  1201.12.1.1
Chef.Bootstrap.WindowsAzure LinuxChefClient  1205.12.2.1
Chef.Bootstrap.WindowsAzure LinuxChefClient  1207.12.3.0
Chef.Bootstrap.WindowsAzure LinuxChefClient  1210.12.4.2000

Can I use my own client.rb if I apply the extension via ARM?

According to the readme, it looks like I can upload the contents of my own client.rb if I apply the extension using PowerShell but I don't see a way to do this via ARM. Is this possible?

Adding "environment" to the bootstrap_options in the ARM template doesn't seem to do anything.

The bootstrap options in my ARM template appears to be ignoring the "environment" property.

"bootstrap_options": {
"environment": "[parameters('chef_env')]",
"chef_node_name": "[parameters('chef_node_name')]",
"chef_server_url": "[parameters('chef_server_url')]",
"validation_client_name": "[parameters('validation_client_name')]"
}

Syntax clarification: runlists

Can there be a syntax clarification on the format of the string to pass for the runlist parameter? I have a bit of confusion .

    "runlist": {
      "value": "recipe[chef-client],recipe[organization-role::default]"
    },

versus

    "runlist": {
      "value": "recipes[chef-client,organization-role::default,organization-role2::special]"
    },

I'm deploying ARM templates via AzureCLI, for context. Silly question as I'm fairly certain it's the former (don't have time to test this quite yet).

How to set the chef run interval frequency

I would like to be able to set the Chef run frequency without having to run a separate cookbook (chef-client) to do so. It would be extremely helpful if the interval and play settings were included in this extension. Is this something that is there which I may have overlooked?

While provisioning with MS Chef extension noticed that var/log/azure/Chef.Bootstrap.WindowsAzure.LinuxChefClient/1210.12.105.1001/chef-client.log is empty

While provisioning with MS Chef extension noticed that var/log/azure/Chef.Bootstrap.WindowsAzure.LinuxChefClient/1210.12.105.1001/chef-client.log is almost empty:

cat /var/log/azure/Chef.Bootstrap.WindowsAzure.LinuxChefClient/1210.12.105.1001/chef-client.log

Logfile created on 2016-08-12 21:51:59 -0500 by logger.rb/53938

Any idea why?

HTTP Request Returned 403 Forbidden for the first boot

Hi,

I am using windows chef extension 1210.12.
OS: 2012R2 with SQL2014
VM size: A2

sometimes, in the first chef boot, I meet the following error:

[2015-09-08T08:39:22+00:00] INFO: Chef Client Service initialized
[2015-09-08T08:39:27+00:00] INFO: Next chef-client run will happen in 0.9096172706210285 seconds
[2015-09-08T08:39:28+00:00] INFO: Reconfiguring with startup parameters
[2015-09-08T08:39:32+00:00] INFO: Chef-Client service is starting a chef-client run...
[2015-09-08T08:39:32+00:00] INFO: Starting chef-client in a new process
[2015-09-08T08:39:36+00:00] WARN: Chef client 1592 is running, will wait for it to finish and then run.
# Logfile created on 2015-09-08 08:39:18 +0000 by logger.rb/41954
[2015-09-08T08:39:58+00:00] INFO: *** Chef 12.4.1 ***
[2015-09-08T08:39:58+00:00] INFO: Chef-client pid: 1592
[2015-09-08T08:40:46+00:00] INFO: Client key c:/chef/client.pem is not present - registering
[2015-09-08T08:40:54+00:00] INFO: HTTP Request Returned 403 Forbidden: error
[2015-09-08T08:40:54+00:00] ERROR: Running exception handlers
[2015-09-08T08:40:54+00:00] ERROR: Report handler AzureExtension::ExceptionHandler raised #<NoMethodError: undefined method `name' for nil:NilClass>
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/azure-chef-extension-0.0.1/lib/chef/azure/chefhandlers/exception_handler.rb:21:in `report'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/handler.rb:226:in `run_report_unsafe'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/handler.rb:214:in `run_report_safely'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/handler.rb:118:in `block in run_exception_handlers'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/handler.rb:117:in `each'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/handler.rb:117:in `run_exception_handlers'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/handler.rb:127:in `block in <class:Handler>'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:428:in `call'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:428:in `block in run_failed'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:427:in `each'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:427:in `run_failed'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:296:in `rescue in run'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:306:in `run'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:252:in `run_with_graceful_exit_option'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:229:in `block in run_chef_client'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/local_mode.rb:39:in `with_server_connectivity'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:212:in `run_chef_client'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/application/client.rb:375:in `run_application'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:60:in `run'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/embedded/apps/chef/bin/chef-client:26:in `<top (required)>'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/bin/chef-client:67:in `load'
[2015-09-08T08:40:54+00:00] ERROR: C:/opscode/chef/bin/chef-client:67:in `<main>'
[2015-09-08T08:40:54+00:00] ERROR: Exception handlers complete
[2015-09-08T08:40:54+00:00] FATAL: Stacktrace dumped to c:/chef/cache/chef-stacktrace.out
[2015-09-08T08:40:54+00:00] FATAL: Net::HTTPServerException: 403 "Forbidden"
[2015-09-08T08:41:05+00:00] INFO: *** Chef 12.4.1 ***
[2015-09-08T08:41:05+00:00] INFO: Chef-client pid: 3672
[2015-09-08T08:41:35+00:00] INFO: Client key c:/chef/client.pem is not present - registering
[2015-09-08T08:41:36+00:00] INFO: HTTP Request Returned 404 Object Not Found: error
[2015-09-08T08:41:37+00:00] INFO: Run List is []
[2015-09-08T08:41:37+00:00] INFO: Run List expands to []
[2015-09-08T08:41:37+00:00] INFO: Starting Chef Run for sqlvm0.AutoRG.net
[2015-09-08T08:41:37+00:00] INFO: Running start handlers
[2015-09-08T08:41:37+00:00] INFO: Loading Handler environment...
[2015-09-08T08:41:37+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000
[2015-09-08T08:41:37+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000\Status/0.status
[2015-09-08T08:41:37+00:00] INFO: Reporting heartbeat...
[2015-09-08T08:41:37+00:00] INFO: Start handlers complete.
[2015-09-08T08:41:37+00:00] INFO: HTTP Request Returned 404 Not Found: 
[2015-09-08T08:41:37+00:00] INFO: Loading cookbooks []
[2015-09-08T08:41:37+00:00] WARN: Node sqlvm0.AutoRG.net has an empty run list.
[2015-09-08T08:41:37+00:00] INFO: Chef Run complete in 0.738975 seconds
[2015-09-08T08:41:37+00:00] INFO: Running report handlers
[2015-09-08T08:41:37+00:00] INFO: Loading Handler environment...
[2015-09-08T08:41:37+00:00] INFO: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000\RuntimeSettings --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000\Status --> C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000\Status\HeartBeat.Json --> C:\WindowsAzure\Logs\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000
[2015-09-08T08:41:37+00:00] INFO: Status file name: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.4.1000\Status/0.status
[2015-09-08T08:41:37+00:00] INFO: Reporting heartbeat...
[2015-09-08T08:41:37+00:00] INFO: Report handlers complete
[2015-09-08T08:41:45+00:00] INFO: Child process exited (pid: 3360)
[2015-09-08T08:41:45+00:00] INFO: Next chef-client run will happen in 1800.0805891492241 seconds
[2015-09-08T09:11:47+00:00] INFO: Reconfiguring with startup parameters
[2015-09-08T09:11:51+00:00] INFO: Chef-Client service is starting a chef-client run...
[2015-09-08T09:11:52+00:00] INFO: Starting chef-client in a new process

Stacktrace file:

Generated at 2015-09-08 08:40:54 +0000
Net::HTTPServerException: 403 "Forbidden"
C:/opscode/chef/embedded/lib/ruby/2.0.0/net/http/response.rb:119:in `error!'
C:/opscode/chef/embedded/apps/chef/lib/chef/http.rb:145:in `request'
C:/opscode/chef/embedded/apps/chef/lib/chef/http.rb:126:in `post'
C:/opscode/chef/embedded/apps/chef/lib/chef/api_client/registration.rb:93:in `create'
C:/opscode/chef/embedded/apps/chef/lib/chef/api_client/registration.rb:84:in `create_or_update'
C:/opscode/chef/embedded/apps/chef/lib/chef/api_client/registration.rb:57:in `run'
C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:608:in `register'
C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:253:in `run'
C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:252:in `run_with_graceful_exit_option'
C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:229:in `block in run_chef_client'
C:/opscode/chef/embedded/apps/chef/lib/chef/local_mode.rb:39:in `with_server_connectivity'
C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:212:in `run_chef_client'
C:/opscode/chef/embedded/apps/chef/lib/chef/application/client.rb:375:in `run_application'
C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:60:in `run'
C:/opscode/chef/embedded/apps/chef/bin/chef-client:26:in `<top (required)>'
C:/opscode/chef/bin/chef-client:67:in `load'
C:/opscode/chef/bin/chef-client:67:in `<main>'

The second boot is OK, but as you know the RUN LIST will expands to empty, and this cause my deployment fail. I deploy 105 VM with chef extension, 7 VM meet this issue.

Please help to check. Thank you.

Support Windows Nano Server

It would be great to use the azure extension to install chef on azure deployed chef Nano Server VMs. Due to the stripped down nature of Nano Server, there are some changes that would need to be made in this repo to accomodate Nano.

Any powershell used must comply with the APIs supported by the .Net core runtime (as opposed to the Full .Net API). If you look at the PR at chef/knife-windows#399 which adds nano support to knife bootstrap in knife-windows. It leverages the mixlib-install gem for all of the chef-client download and install scripts. You may not be able to use that gem directly here because you need a ruby runtime but you could use that gem as a reference creating the scripts.
You need an appx package to install chef on Nano. In knife bootstrap one currently has to pass a custom URL to --msi-url that points to a chef-client appx. An url that currently works is https://s3-us-west-2.amazonaws.com/nano-chef-client/chef-12.14.60.appx I'm not familiar enough with the extension to know if that is possible - to pass a custom url. If not we may need to wait for appx packages to be delivered from Omnitruck.

install on windows not respecting proxy config

https://github.com/chef-partners/azure-chef-extension/blob/master/ChefExtensionHandler/bin/chef-install.psm1

line: 44

I needs to respect the proxy setup at the system level.

Maybe doing something like this:

        $proxyAddr = (get-itemproperty 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings').ProxyServer
        $proxy = new-object System.Net.WebProxy
        $proxy.Address = $proxyAddr
        $proxy.useDefaultCredentials = $true

        $wc = new-object system.net.WebClient
        $wc.proxy = $proxy

        iex $wc.DownloadString('https://omnitruck.chef.io/install.ps1');install

Chef extension is installing version 12.5.1. How is this possible?

If the latest release of this extension is 12.10.12.4.1000, why is version 12.5.1 of the chef-client being installed?

Extension uninstallation risks should be reduced

Uninstalling the extension currently uninstalls chef-client, and uninstall happens even when users are not aware, specifically when a new minor version of the extension is released, which is fairly often. This is because Azure interprets minor version updates as a time to update the extension, and "update" is implemented by having the Azure agent disable the extension, uninstall it, and then install it (those "commands" are essentially part of the agent - extension protocol, so the extension simply carries out instructions given by Azure).

In cases where there are race conditions between uninstall and install (often caused by a combination of slow Windows VM's and the generally slow install / uninstall time of chef-client), this can lead to a case where chef-client gets uninstalled and can't be reinstalled during an update triggered by a deployment of the extension (users don't deploy the extension, this happens all the time without their knowledge as Chef updates it to add features). This means that suddenly and without warning, systems can end up in a state where chef-client is uninstalled and the systems are completely unmanageable.

A workaround is to re-add the extension, possibly after a reboot.

We should minimize this possibility with the following plan:

Phase 1 -- don't uninstall

We should change the extension in the following way:

Remove / comment out the line that calls msiexec / dpkg, etc. to uninstall the chef-client -- this makes uninstall a no-op, and has the downside of leaving chef-client behind
Verify that if an extension uninstall occurs, the binaries are still there, but the service is disabled.
Verify that after an update we see a success and that chef-client service is running successfully.
Verify all of the above on Windows, CentOS, Ubuntu

Phase 2 -- implement delayed uninstall for the update case

To restore the old functionality of uninstall during update, which is a cleaner approach (it gets rid of native gems for instance), make the following changes

During unininstall, write some state (e.g. a file) indicating that an uninstall was requested, otherwise do nothing else
During install, look for that file -- if it's there, delete it, and then perform an uninstall of chef-client, and then proceed with normal install
Verify the update and uninstall cases (uninstall outside the context of update should still not remove chef-client). The failure modes here include failing to update, which is not catastrophic.
Note: we may skip this phase -- we are doing a new installer which is faster and also deletes the chef-client subdirectory, so the uninstall during update may actually be of low value moving forward and not worth this testing / maintenance.

Phase 3 -- add an "uninstall chef-client" flag

This flag would cause us to uninstall chef-client when requested by Azure -- we'd document that it can cause a risk of loss of service since Azure calls uninstall during extension updates. This would allow users with a strict need to uninstall chef-client and leave behind no trace to do so, while minimizing the risk for mainstream use.

Note that another possibility should also be considered in addition to the above options: we could simply engage in patch updates from now on and not increment the minor version. This would prevent Azure from updating chef-client. However, a downside is that we'd have infrequent testing of what happens with minor version updates, and if we did need to rev the major or minor version, we'd push the risk to that scenario.

Make installation of the Chef Windows service optional

The recommendation is to now run the chef-client as a scheduled task rather than as a Windows Service.

Why Run Chef with the Task Scheduler vs. the Windows Service?

More scheduling options - task scheduler can run configurable cycles (think maintenance windows)
Can run on demand
Does not lock the log file between chef runs (for log rotation)
Better visibility into when Chef is actually running on the box and what the last exit code was.
The chef service is buggy - in a way that is really hard to reproduce and therefore fix.

The Chef service is just a less resilient, less transparent version of task scheduler. In order to be sure that chef is running, you'll have to watch either log files or check-ins to the server. The task scheduler gives us better options and is the Microsoft recommended way to run a command that needs to run on a schedule or in response to certain conditions. If you don't have something that has to run all the time in the background or listen for incoming connections, you should not be a service.

The extension should have a new option for installing the optional service from the MSI, which will be false by default, something like install_service: true|false. This is a breaking change and the new option is to support people that still rely on the service but it has to be explicitly enabled. However eventually this option will be removed as well.

The chef-client cookbook does not support the windows service any more just the scheduled task. It is recommended that people include the cookbook in there initial run using the extension so that they end up with a system that is being managed by Chef.

This would also address the issue raised by @aarsan in issue #165.

/cc @smurawski

Cannot download ChefClient - no internet connectivity from our VMs

We have been using this plugin for about a year, I tried to use it for a new VM today. It failed because our servers have no outbound internet connectivity. The previous version worked because the ChefClient was included in the plugin package.

I can see the code change in Install-ChefClient now tries to download the ChefClient from http://www.chef.io.

What do you suggest?

Chef extension consistently fails on Ubuntu VM

For the past few days, I have been unable to successfully enable the Chef extension on my VM. I receive the following error:

private key does not match certificate - Check log file for details

There is a similar issue posted here:

Azure/azure-linux-extensions#28

Please add a client location parameter

It would be very helpful if I could host my own client in blob storage and reference that location in the ARM template.

root crontab gets populated with chef-client line after reboot

After we successfully ran MS chef extension (RHEL7.2 VM), it populates root crontab with below line.Even if we remove that line from crontab, reboot brings it back.

[cwarfiel@cwarfiel cron]$ cat root
# Chef Name: azure_chef_extension
*/30 * * * * /bin/sleep 0; chef-client -c /etc/chef/client.rb -L /var/log/azure/Chef.Bootstrap.WindowsAzure.LinuxChefClient/1210.12.104.1000/chef-client.log --pid /etc/chef/azure-chef-client.pid --once

Is there a way to control that and keep it out of the crontab

Wrong xml format and value of Get-Deployment by using knife-azure to create a VM

https://msdn.microsoft.com/en-us/library/azure/ee460804.aspx

We can get the right format and value of Azure REST api (Get-Deployment) if created the VM by Azure.
By Powershell to get Ge-Deployment :

By Azure JAVA SDK to get Get-Deployment :

However, we found it failed with wrong format and value if created by knife-azure.
By Powershell to get Ge-Deployment :

By Azure JAVA SDK to get Ge-Deployment :

The wrong format of Get-Deployment created by knife-azure:
chris-test1Productionfed1879427af455898376c545a779d5fSuspendedY2hyaXMtOTUyLTI=http://chris-test1.cloudapp.net/PFNlcnZpY2VDb25maWd1cmF0aW9uIHhtbG5zOnhzZD0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEiIHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiIHhtbG5zPSJodHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL1NlcnZpY2VIb3N0aW5nLzIwMDgvMTAvU2VydmljZUNvbmZpZ3VyYXRpb24iPg0KICA8Um9sZSBuYW1lPSJjaHJpcy05NTItMiI+DQogICAgPEluc3RhbmNlcyBjb3VudD0iMSIgLz4NCiAgPC9Sb2xlPg0KPC9TZXJ2aWNlQ29uZmlndXJhdGlvbj4=chris-test1chris-test1StoppedDeallocatedStopped6904E4DFCC84CEBEFBB265E532BB25112987A9CC1Rolei:type="PersistentVMRole"chris-test1PersistentVMRoleNetworkConfiguration5986PowerShell5986tcpfalse3389Remote Desktop3389tcpfalseReadWritechris-test1-chris-test1-0-201505050830580485https://portalvhdsdqnt01flqwh9p.blob.core.windows.net/vhds/chris-test1-chris-test1-2015-05-05.vhda699494373c04fc0bc8f2bb1389d6106__Win2K8R2SP1-Datacenter-201504.01-en.us-127GB.vhdWindowsStandard_D1EAEA705B4F65B8A286B1CE95CF9DD58AA77FF553truefalsefalse2015-05-05T08:30:53Z2015-05-05T08:53:16Zchris-test1.g6.internal.cloudapp.net

Secret parameter should be retrieved from Protected Settings

When using a secret for an encrypted data bag it can be passed as a securestring in the parameters, but the extension takes the value from the bootstrap_options and not protectedSettings.

https://github.com/chef-partners/azure-chef-extension/blob/master/lib/chef/azure/commands/enable.rb#L171

Could this be changed so that it is specified in protectedSettings in the ARM template?

Error creating Ubuntu VM in ARM mode: typeHandlerVersion is invalid

I'm trying to create a Ubuntu Linux VM in ARM mode with knife azurerm but I get the following error
The value of parameter typeHandlerVersion is invalid. (Code: InvalidParameter)
Any idea what a valid parameter is here?

Extension metadata has incorrect url for Chef

This file contains references to the DNS name getchef.com, but that is no longer Chef's DNS name. This should be changed to chef.io.

Unexpected release of Chef VM Extension 1210.12.5.1000 means no way to build machines with ChefClient 12.4.x

I can see the new version of the extension 1210.12.5.1000 has been deployed which decouples the extension from the ChefClient payload:

Type             Version
----             -------
CentosChefClient 11.12.4.2
CentosChefClient 11.14.6.1
ChefClient       11.10.4
ChefClient       11.12.0.0
ChefClient       11.18.6.2
ChefClient       1205.12.2.1
ChefClient       1207.12.3.0
ChefClient       1210.12.4.1000
ChefClient       1210.12.5.1000
LinuxChefClient  11.18.6.2
LinuxChefClient  1201.12.1.1
LinuxChefClient  1205.12.2.1
LinuxChefClient  1207.12.3.0
LinuxChefClient  1210.12.4.2000
LinuxChefClient  1210.12.5.1000

However, in an ARM template it is not possible to specify the build and revision number hence the latest build is still specified as version '1210.12'. The net result is that the most recent release of the VM extension means the non-decoupled style of deployment can no longer be achieved for new machines, and therefore I can no longer build a machine with ChefClient 12.4.x

Suggestion:

Yank the 1210.12.5.1000 release so that the behaviour seen is documented/expected
Build a new VM Extension with a new Major version number (e.g. something like 1251.0 or anything higher than 1210) so it:

requires people to choose whether to upgrade to it rather than being forced to take it if they rebuild their machines using the same template
differentiates this build of the extension from the previous 1210.12 build

Parameterise the ChefClient build number so it can be passed to OmniTruck and people can override it in publicSettings.config - unless I am reading https://github.com/chef-partners/azure-chef-extension/blob/master/ChefExtensionHandler/bin/chef-install.psm1#L81-L82 incorrectly?

Related: #81 #75

Installation intermittently fails with "An exception occurred during a WebClient request" during install phase

Intermittently (using ARM) I get an error that looks similar to seen by #114 (/cc @aarsan). It seems that intermittently the machine cannot reach Omnitruck or there's some other failure to download between all the redirections that happen (packages.chef.io -> akamai/bintray etc.), causing the Chef Client not to get installed (despite the subsequent message which says it is installed successfully). Subsequent calls to gem install the Azure plugin obviously fail.

Manifestation in Azure deployment log:

{
  "status": "Failed",
  "error": {
    "code": "ResourceDeploymentFailure",
    "message": "The resource operation completed with terminal provisioning state 'Failed'.",
    "details": [
      {
        "code": "VMExtensionHandlerNonTransientError",
        "message": "Handler 'Chef.Bootstrap.WindowsAzure.ChefClient' has reported failure for VM Extension 'chefExtension' with terminal error code '1007' and error message: 'Install failed for plugin (name: Chef.Bootstrap.WindowsAzure.ChefClient, version 1210.12.102.1000) with exception Command C:\\Packages\\Plugins\\Chef.Bootstrap.WindowsAzure.ChefClient\\1210.12.102.1000\\install.cmd of Chef.Bootstrap.WindowsAzure.ChefClient has exited with Exit code: 1'"
      }
    ]
  }
}

On the node itself (CommandExecution log):

[04/06/2016 12:01:13.41] Executing: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\install.cmd 
[04/06/2016 12:01:31.27] Execution Complete.
######
Execution Output:
C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000>set CHEF_EXT_DIR=C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\ 
C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000>echo C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\ 
C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\
C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000>powershell -nologo -noprofile -executionpolicy unrestricted Import-Module C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\bin\chef-install.psm1;Install-ChefClient 
Downloading Chef Client ...
Error running install: Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
[04/06/2016 12:01:23] Installing chef...
Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName          
-------  ------    -----      ----- -----   ------     -- -----------          
      1       2      168        492     4     0.02   2280 msiexec              
[04/06/2016 12:01:24] Chef Client Package installed successfully!
[04/06/2016 12:01:24] Installing Azure-Chef-Extension gem
The term 'gem' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.


Execution Error:

######
Command C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\install.cmd of Chef.Bootstrap.WindowsAzure.ChefClient has exited with Exit code: 1

Here's what the process looks like on a good machine things look like this (log taken minutes earlier - it's another machine in the same template):

[04/06/2016 11:57:00.80] Executing: C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\install.cmd 
[04/06/2016 11:58:48.93] Execution Complete.
######
Execution Output:
C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000>set CHEF_EXT_DIR=C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\ 
C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000>echo C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\ 
C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\
C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000>powershell -nologo -noprofile -executionpolicy unrestricted Import-Module C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\bin\chef-install.psm1;Install-ChefClient 
Downloading Chef Client ...
[04/06/2016 11:57:14] Installing chef...
Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName          
-------  ------    -----      ----- -----   ------     -- -----------          
      1       2      164        484     4     0.00   2484 msiexec              
[04/06/2016 11:58:38] Chef Client Package installed successfully!
[04/06/2016 11:58:38] Installing Azure-Chef-Extension gem
Successfully installed azure-chef-extension-0.0.1
1 gem installed
[04/06/2016 11:58:48] Installed Azure-Chef-Extension gem successfully


Execution Error:

######
Command C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.102.1000\install.cmd of Chef.Bootstrap.WindowsAzure.ChefClient has exited with Exit code: 0
Plugin (name: Chef.Bootstrap.WindowsAzure.ChefClient, version: 1210.12.102.1000) completed successfully.

Azure extension enable command fails as it takes more that 5 minutes at times

How do i get the newest version on Windows VMs?

No matter what I try, I end up with version 12.3 of the Chef client when I deploy with an ARM template. The quickstarts from MS have a VM with Chef client installed, but the "autoupdate" value is never used in the template, so I don't know where to put it. Also, I have no idea where to tell it the version of the client to use...is it "typeHandlerVersion"?

Here's what I have used and it's yet to really work the way I'd expect. Nothing I see lets me pick the most recent version of the client, nor is there anything in there to update automatically.

   {
        "type": "Microsoft.Compute/virtualMachines/extensions",
        "name": "[concat(parameters('vmNameBase'),copyIndex(1),'/ChefClient')]",
        "apiVersion": "2015-06-15",
        "location": "[variables('location')]",
        "copy": {
            "name": "vmCopy",
            "count": "[parameters('vmCount')]"
        },
        "dependsOn": [
            "[concat('Microsoft.Compute/virtualMachines/', parameters('vmNameBase'),copyIndex(1))]"
        ],
        "properties": {
            "publisher": "Chef.Bootstrap.WindowsAzure",
            "type": "ChefClient",
            "typeHandlerVersion": "1207.12",
            "settings": {
                "bootstrap_options": {
                    "chef_node_name": "[concat(parameters('vmNameBase'),copyIndex(1))]",
                    "chef_server_url": "[parameters('chef_server_url')]",
                    "validation_client_name": "[parameters('validation_client_name')]"
                },
                "runlist": "[parameters('runlist')]"
            },
            "protectedSettings": {
                "validation_key": "[parameters('validation_key')]"
            }
        }
    }

Passing secret for decrypting data bags is not working

Hey Team!

I have been trying to use the secret bootstrap option with no success. I have tried to put the secret key in the template file itself as a string or passing it on the command line, but no luck. What I noticed is that the actual secret file itself is UTF-8 but when I look at the file that gets created by the azure-chef-extension, it appears to be ANSI encoded. This is happening whether I try to bootstrap a Linux or Windows VM.

I could certainly be doing something wrong, but the doc isn't too detailed on how I need to pass the string.

Thanks so much for the extension!

bootstrap_options environment setting is not working

I am using Azure Powershell to provision the Chef Windows extension 1207.12.3.0 using the following command:

Set-AzureVMChefExtension -ValidationPem C:\xxxx.pem `
                         -ValidationClientName "xxxx-validator" `
                         -ChefServerUrl "https://api.opscode.com/organizations/xxxxx" `
                         -Windows `
                         -AutoUpdateChefClient `
                         -RunList "role[xxxx]"  `
                         -BootstrapOptions "{`"environment`": `"env_name_here`"}"

It seems the machine never gets set to the correct environment.

The C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1207.12.3.0\RuntimeSettings\0.settings file on the vm looks like this:

{
  "runtimeSettings": [
    {
      "handlerSettings": {
        "protectedSettingsCertThumbprint": "XXXXX",
        "protectedSettings": "XXXXXXXXXX",
        "publicSettings": {
          "autoUpdateClient": "true",
          "deleteChefConfig": "false",
          "client_rb": "xxxxxxxxxxxx",
          "runlist": "\"role[xxxx]\"",
          "bootstrap_options": {
            "environment": "env_name_here"
          }
        }
      }
    }
  ]
}

Documentation to use PowerShell to Setup VM (CentOS)

Is there somewhere an documentation how to setup an new Linux VM via PowerShell or AZURE-CLI with Chef-Extension enabled ?

Regarding this issue: #17
CentOS is now supported? Would be great if someone could provide some kind of documentation.

Validation key is not nested properly in 0.settings file

Using the Chef Extension in an ARM template is generating an invalid 0.Settings file where the validation key is not nested properly in protectedSettings.

The ChefExtension is being configured using the following template:

{
      "name": "[concat(parameters('vmName'),'/ChefExtension')]",
      "type": "Microsoft.Compute/virtualMachines/extensions",
      "location": "[resourceGroup().location]",
      "apiVersion": "2015-06-15",
      "properties": {
        "publisher": "Chef.Bootstrap.WindowsAzure",
        "type": "ChefClient",
        "typeHandlerVersion": "1210.12",
        "settings": {
          "validation_key_format": "base64encoded",
          "bootstrap_options": {
            "chef_node_name": "[parameters('vmName')]",
            "chef_server_url": "[parameters('chefServerUrl')]",
            "validation_client_name": "[parameters('chefValidatorName')]",
            "node_ssl_verify_mode": "none"
          },
          "runlist": "[parameters('chefRunlist')]"
        },
        "protectedSettings": {
          "validation_key": "[parameters('chefValidatorKey')]"
        }
      }
    }

This generates the following 0.settings file, which as can be seen has the key as the protectedSettings with no validation_key sub setting:

{
  "runtimeSettings": [
    {
      "handlerSettings": {
        "protectedSettingsCertThumbprint": "D962D58667B0AB8B4CAAE669BE6139DBC40DC7D3",
        "protectedSettings": "MIITRAYJKoZIhvc...HFA9RK4pVow==",
        "publicSettings": {
          "validation_key_format": "base64encoded",
          "bootstrap_options": {
            "chef_node_name": "srh-node-1",
            "chef_server_url": "https://someserver123.northeurope.cloudapp.azure.com",
            "validation_client_name": "microsoft-validator",
            "node_ssl_verify_mode": "none"
          },
          "runlist": "recipe[awesome_customers_windows]"
        }
      }
    }
  ]
}

This makes the initial run fail as the c:\chef\validation.pem does not exist as it has not been extracted.

I have only tested this on Windows myself, I do not know if this is causing an issue on Linux but I suspect that it might.

/cc @jjasghar

chef-client decrypt error with multiple certs/keys in Certificates.pem bundle

Hi there, when a VM is deployed with SSH key authentication, the Certificate.pem bundle contains multiple certificates, only one of which is able to decrypt the privateConfig container. this means the chef-client enable command fails with decrypt error.

I have updated enable.rb locally with additional code, which loops through all certificate/key pairs in the bundle and attempts to decrypt. This seems to work.

I am not a Ruby guy, so my code may not be the best example. Having said that, would you like me to commit directly to the main branch, branch off, or would you rather update the relevant code on your own?

Here is my update, lib/chef/azure/commands/enable.rb:260:

  ...
  certificate_path = LINUX_CERT_PATH

  # read certs & keys from the certificate bundle and attempts to decrypt
  cert_regex = /-----BEGIN CERTIFICATE-----(.*?)-----END CERTIFICATE-----/m
  key_regex = /-----BEGIN PRIVATE KEY-----(.*?)-----END PRIVATE KEY-----/m
  begin_cert_line = "-----BEGIN CERTIFICATE-----"
  end_cert_line = "-----END CERTIFICATE-----"
  begin_pri_key_line = "-----BEGIN PRIVATE KEY-----"
  end_pri_key_line = "-----END PRIVATE KEY-----"

  fc = File.read(certificate_path)
  cert_matches = fc.scan(cert_regex)
  key_matches = fc.scan(key_regex)
  key_dict = {}
  key_dict['certs'] = cert_matches
  key_dict['keys'] = key_matches

  i = 0
  enc_text = encrypted_text
  while i < key_dict.count
    cert = "%s\n%s\n%s\n" % [begin_cert_line, key_dict['certs'][i].join.strip, end_cert_line]
    key = "%s\n%s\n%s\n" % [begin_pri_key_line, key_dict['keys'][i].join.strip, end_pri_key_line]
    puts "Processing certificate %i of %i..." % [i+1, key_dict.count]
    certificate = OpenSSL::X509::Certificate.new cert
    private_key = OpenSSL::PKey::RSA.new key
    encrypted_text = Base64.decode64(enc_text)
    encrypted_text = OpenSSL::PKCS7.new(encrypted_text)
    begin
      decrypted_text = encrypted_text.decrypt(private_key, certificate)       
      puts "Processed certificate %i of %i" % [i+1, key_dict.count]
    rescue
      puts "Error processing certificate %i of %i" % [i+1, key_dict.count]
    end
    i+=1
  end
end

#extract validation_key from decrypted hash
...

Please let me know how you'd like to proceed..

-- ab1

azure-chef-extension v1210.12 is not available for West US location

When I deploy v1210.12 chef extension to VM (windows) in West US location. It reports the following error:

New-AzureResourceGroupDeployment : 5:57:05 AM - Resource Microsoft.Compute/virtualMachines/extensions
'myVM/chefExtension' failed with message 'Extension with publisher 'Chef.Bootstrap.WindowsAzure', type 'ChefClient',
and type handler version '1210.12' could not be found in the extension repository.'
At line:1 char:1
+ New-AzureResourceGroupDeployment -Name $deployName -ResourceGroupName $RGName -T ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-AzureResourceGroupDeployment], Exception
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Resources.NewAzureResourceGroupDeploymentCommand

Only 1207.12 version is workable in West US location.

The same template(v1210.12) is OK for VM in East Asia.

Azure extension is impractical when bootstrapping servers with lengthy run lists

I have found the new Azure client extension an interesting though ultimately frustrating way of bootstrapping Azure VMs, because it looks like Microsoft's agent enforces a 5 minute timeout for any agent plugin to complete. After this timeout, it kills the plugin:

[00000004] [01/20/2015 02:54:32.29] [INFO]  Staring a process with the launch command C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\11.16.4.2\enable.cmd and params:
[00000004] [01/20/2015 02:54:32.29] [INFO]  Beginning enable of plugin Chef.Bootstrap.WindowsAzure.ChefClient.
[00000004] [01/20/2015 02:54:32.29] [INFO]  Waiting for command C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\11.16.4.2\enable.cmd of plugin (name Chef.Bootstrap.WindowsAzure.ChefClient version 11.16.4.2) to finish...
[00000004] [01/20/2015 02:59:35.32] [ERROR] Command C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\11.16.4.2\enable.cmd of Chef.Bootstrap.WindowsAzure.ChefClient has not exited on time! Killing it...
[00000004] [01/20/2015 02:59:35.32] [ERROR] Enable command of Chef.Bootstrap.WindowsAzure.ChefClient has exited with Exit code: -2
[00000004] [01/20/2015 02:59:35.33] [WARN]  Enable failed for plugin (name: Chef.Bootstrap.WindowsAzure.ChefClient, version 11.16.4.2) with exception Command C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\11.16.4.2\enable.cmd of
 Chef.Bootstrap.WindowsAzure.ChefClient has not exited on time! Killing it..., Code: 1009
[00000004] [01/20/2015 02:59:35.47] [ERROR] Error while processing plugin Chef.Bootstrap.WindowsAzure.ChefClient version 11.16.4.2, state: enabled, error: System.Exception: Command C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\11.16.4.2\enable.cmd of Chef.Bootstrap.WindowsAzure.ChefClient has not exited on time! Killing it...
   at Microsoft.WindowsAzure.GuestAgent.ContainerStateMachine.HandlerStateMachine.InvokeCommand(String command, PluginArtifacts pluginArtifact, String pluginVersion, String pluginFolder, String pluginLogFolder, Int32 processWaitTimeout, PluginEventType startType, PluginEventType endType)
   at Microsoft.WindowsAzure.GuestAgent.ContainerStateMachine.HandlerStateMachine.EnableHandler(PluginArtifacts artifact)
   at Microsoft.WindowsAzure.GuestAgent.ContainerStateMachine.HandlerStateMachine.StartHandlerStateMachine(PluginArtifacts data, RDConfigHostingEnvironmentSettingsPlugin plugin)
   at Microsoft.WindowsAzure.GuestAgent.ContainerStateMachine.PluginInstaller.PluginBackgroundThread(Object sender, DoWorkEventArgs e)

This "feature" makes the bootstrapping of servers with anything more than a trivial run_list to be totally impractical because it is common for an initial Chef run to last more than 5 minutes.

Could we talk with MSFT about this limit?

chef cronjob disabled in some VMs

Hi,

it looks like there was some issue last night with an attempted update to 1210.12.104.1000 that disabled chef in some of our VMs.

waagent.log

2016/06/21 01:38:10 Retrieved GoalState from Azure Fabric.
2016/06/21 01:38:10 ExpectedState: Started
2016/06/21 01:38:10 ContainerId: [snip]
2016/06/21 01:38:10 RoleInstanceId: [snip]
2016/06/21 01:38:10 Private key with thumbprint: [snip] was retrieved.
2016/06/21 01:38:10 Public cert with thumbprint:  [snip] was retrieved.
2016/06/21 01:38:10 Posted Role Properties. CertificateThumbprint= [snip]
2016/06/21 01:38:13 Found Plugin: Chef.Bootstrap.WindowsAzure.LinuxChefClient version: 1210.12.104.1000
2016/06/21 01:38:13 Downloading plugin manifest: Chef.Bootstrap.WindowsAzure.LinuxChefClient from  [snip]
2016/06/21 01:38:13 Plugin server is:  [snip]
2016/06/21 01:38:13 Bundle URI =  [snip]
2016/06/21 01:38:13 Plugin bundle [snip] successfully length = 75695
2016/06/21 01:38:13 Extracted  [snip]/Chef.Bootstrap.WindowsAzure__LinuxChefClient__1210.12.104.1000 to /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000
2016/06/21 01:38:13 Found RuntimeSettings for Chef.Bootstrap.WindowsAzure.LinuxChefClient V 1210.12.104.1000
2016/06/21 01:38:13 SetHandlerState: Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000, NotInstalled
2016/06/21 01:38:13 Spawned disable.sh PID 34936
2016/06/21 01:38:18 disableCommand completed.
2016/06/21 01:38:18 SetHandlerState: Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.103.1000, Disabled
2016/06/21 01:38:18 Chef.Bootstrap.WindowsAzure.LinuxChefClient version 1210.12.103.1000 is disabled
2016/06/21 01:38:18 Copy status file from old plugin dir to new
2016/06/21 01:38:18 Spawned update.sh PID 35166
2016/06/21 01:38:23 ERROR:Process 35166 returned non-zero exit code(1)
2016/06/21 01:38:23 ERROR:Update failed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000
2016/06/21 01:38:23 Spawned uninstall.sh PID 35201
2016/06/21 01:38:28 uninstallCommand completed.
2016/06/21 01:38:28 SetHandlerState: Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.103.1000, NotInstalled
2016/06/21 01:38:28 Uninstall completeChef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.103.1000
2016/06/21 01:38:28 Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.103.1000 extension files deleted.
2016/06/21 01:38:28 Found RuntimeSettings for Chef.Bootstrap.WindowsAzure.LinuxChefClient V 1210.12.104.1000
2016/06/21 01:38:28 Spawned install.sh PID 35221
2016/06/21 01:38:33 installCommand completed.
2016/06/21 01:38:33 SetHandlerState: Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000, Installed
2016/06/21 01:38:33 Installation completed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000
2016/06/21 01:38:33 Spawned enable.sh PID 35241
2016/06/21 01:38:38 enableCommand completed.
2016/06/21 01:38:38 SetHandlerState: Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000, Enabled
2016/06/21 01:38:38 Enable completed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000
2016/06/21 01:38:38 Processing completed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000
2016/06/21 01:38:38 Found Plugin: Microsoft.OSTCExtensions.LinuxDiagnostic version: 2.3.9001
2016/06/21 01:38:38 Found RuntimeSettings for Microsoft.OSTCExtensions.LinuxDiagnostic V 2.3.9001
2016/06/21 01:38:38 Spawned diagnostic.py -enable PID 35258
2016/06/21 01:38:39 LinuxAzureDiagnostic started to handle.
2016/06/21 01:38:39 [Microsoft.OSTCExtensions.LinuxDiagnostic-2.3.9001] cwd is /var/lib/waagent/Microsoft.OSTCExtensions.LinuxDiagnostic-2.3.9001
2016/06/21 01:38:39 [Microsoft.OSTCExtensions.LinuxDiagnostic-2.3.9001] Change log file to /var/log/azure/Microsoft.OSTCExtensions.LinuxDiagnostic/2.3.9001/extension.log
2016/06/21 01:38:43 enableCommand completed.

And the extension log:
/var/log/azure/Chef.Bootstrap.WindowsAzure.LinuxChefClient/1210.12.104.1000/CommandExecution.log

2016/06/21 01:38:13 Downloading plugin manifest: Chef.Bootstrap.WindowsAzure.LinuxChefClient from [snip]/Chef.Bootstrap.WindowsAzure_LinuxChefClient_europenorth_manifest.xml
2016/06/21 01:38:13 Plugin server is: rdfepirv2db3prdstr02.blob.core.windows.net
2016/06/21 01:38:13 Bundle URI = [snip]/Chef.Bootstrap.WindowsAzure__LinuxChefClient__1210.12.104.1000
2016/06/21 01:38:13 Plugin bundle [snip]/Chef.Bootstrap.WindowsAzure__LinuxChefClient__1210.12.104.1000downloaded successfully length = 75695
2016/06/21 01:38:13 Extracted [snip]/Chef.Bootstrap.WindowsAzure__LinuxChefClient__1210.12.104.1000 to /var/lib/waagent/Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000
2016/06/21 01:38:13 Found RuntimeSettings for Chef.Bootstrap.WindowsAzure.LinuxChefClient V 1210.12.104.1000
2016/06/21 01:38:13 Spawned disable.sh PID 34936
2016/06/21 01:38:18 disableCommand completed.
2016/06/21 01:38:18 Chef.Bootstrap.WindowsAzure.LinuxChefClient version 1210.12.103.1000 is disabled
2016/06/21 01:38:18 Spawned update.sh PID 35166
2016/06/21 01:38:23 Process 35166 returned non-zero exit code(1)
2016/06/21 01:38:23 Update failed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000
2016/06/21 01:38:23 Spawned uninstall.sh PID 35201
2016/06/21 01:38:28 uninstallCommand completed.
2016/06/21 01:38:28 Uninstall completeChef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.103.1000
2016/06/21 01:38:28 Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.103.1000 extension files deleted.
2016/06/21 01:38:28 Found RuntimeSettings for Chef.Bootstrap.WindowsAzure.LinuxChefClient V 1210.12.104.1000
2016/06/21 01:38:28 Spawned install.sh PID 35221
2016/06/21 01:38:33 installCommand completed.
2016/06/21 01:38:33 Installation completed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000
2016/06/21 01:38:33 Spawned enable.sh PID 35241
2016/06/21 01:38:38 enableCommand completed.
2016/06/21 01:38:38 Enable completed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000
2016/06/21 01:38:38 Processing completed for Chef.Bootstrap.WindowsAzure.LinuxChefClient-1210.12.104.1000

This left the system in a state where the chef-client is no longer being run in the crontab, and hasn't checked in with the Chef server since that event.

There were several other machines that didn't try to update, but I'm not sure how they might be configured differently. Is there a way to block this kind of update from happening?

Can't bootstrap nodes with on-prem Chef server

Hi there,

I've successfully used this extension to bootstrap Windows nodes via ARM template when using Hosted Chef, however with my own on-prem chef server, I am getting the following errors on clients when deploying the same way as I did with Hosted Chef:

C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.105.1000>set CHEF_EXT_DIR=C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.105.1000\ C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.105.1000>echo C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.105.1000\ C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.105.1000\ C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.105.1000>set path=C:\opscode\chef\bin;C:\opscode\chef\embedded\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\; C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.105.1000>ruby C:\Packages\Plugins\Chef.Bootstrap.WindowsAzure.ChefClient\1210.12.105.1000\bin\chef-enable.rb 2016-07-27 15:05:25 +0000 extension_root --> C:/Packages/Plugins/Chef.Bootstrap.WindowsAzure.ChefClient/1210.12.105.1000 2016-07-27 15:05:25 +0000 Creating EnableChef object with []... 2016-07-27 15:05:25 +0000 Running Chef extension enable command... Bootstrap directory [C:/chef] already exists, skipping creation... 2016-07-27 15:05:25 +0000 Creating chef configuration files [2016-07-27T15:05:26+00:00] ERROR: Chef client key parsing error. #<OpenSSL::PKey::RSAError: Neither PUB key nor PRIV key: header too long> [2016-07-27T15:05:26+00:00] ERROR: Chef Server SSL certificate parsing error. #<OpenSSL::X509::CertificateError: header too long> 2016-07-27 15:05:26 +0000 Created chef configuration files 2016-07-27 15:05:26 +0000 Launching chef-client to register node with the runlist 2016-07-27 15:05:26 +0000 Successfully launched chef-client process with PID [1052] 2016-07-27 15:05:26 +0000 Getting chef-client service status 2016-07-27 15:05:26 +0000 Installing chef-client service... 2016-07-27 15:05:32 +0000 Installed chef-client service. Logging status message: chef-service installed 2016-07-27 15:05:32 +0000 Starting chef-client service... 2016-07-27 15:05:37 +0000 Started chef-client service. Logging status message: chef-service enabled

Further, this is my chef-stacktrace.out file:

OpenSSL::SSL::SSLError: SSL Error connecting to https://onpremchefserver/organizations/cheforg/clients - SSL_connect returned=1 errno=0 state=error: certificate verify failed C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/http.rb:376:inrescue in retrying_http_errors'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/http.rb:333:in retrying_http_errors' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/http.rb:299:insend_http_request'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/http.rb:144:in request' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/http.rb:127:inpost'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/api_client/registration.rb:96:in create' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/api_client/registration.rb:87:increate_or_update'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/api_client/registration.rb:58:in run' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/client.rb:620:inregister'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/client.rb:266:in run' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/application.rb:268:inrun_with_graceful_exit_option'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/application.rb:244:in block in run_chef_client' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/local_mode.rb:44:inwith_server_connectivity'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/application.rb:227:in run_chef_client' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/application/client.rb:456:insleep_then_run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/application/client.rb:443:in block in interval_run_chef_client' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/application/client.rb:442:inloop'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/application/client.rb:442:in interval_run_chef_client' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/application/client.rb:426:inrun_application'
C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/lib/chef/application.rb:59:in run' C:/opscode/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.12.15-universal-mingw32/bin/chef-client:26:in<top (required)>'
C:/opscode/chef/bin/chef-client:63:in load' C:/opscode/chef/bin/chef-client:63:in

I am able to successfully bootstrap a Windows server from my Chef workstation with no issue, and I've made sure the validation.pem key that gets placed in c:\chef is identical on successfully bootstrapped clients and unsuccessfully bootstrapped clients.

Looking forward to some help!
Jon