Error message intermittently received during knife-azure deployment - began Wed PM Dec 2nd:
"FATAL: Unable to create role:BadRequest : The certificate with thumbprint ____ was not found"

We can issue the same exact command multiple times - just minutes from each other. Sometimes error will occur, sometimes command will successfully complete - it seems to be random. The workaround right now is - if anyone gets this error message, just re-submit same exact command and hope it goes through. This definitely impacts ability to use knife-azure for reliable end-to-end environmental deployments.

In -VV debug mode, we notice the same thumbprint given in the error message is EXACTLY SAME as what's submitted XML in the "PublicKeys" session for authorized_keys :

<PublicKeys>
      <PublicKey>
        <Fingerprint>SAME THUMBPRINT AS ERROR</Fingerprint>
        <Path>/home/devops/.ssh/authorized_keys</Path>
      </PublicKey>
    </PublicKeys>

This issue began ~ Wednesday Dec 2nd evening time (PST) - we happened to notice same timeframe the new Azure Portal went GA. Is it possible something was updated on the Azure API that reduced backwards compatibility with the format these SSH key thumbprints are being submitted by the knife-azure tool?

Hi

I am using the open source version-11 chef server/knife/client all on ubuntu machines. What version of chef will work with the knife-azure plugin?

regards

meher

From my workstation, using knife azure server create and the 2008r2 image, I am unable to successfully bootstrap the node to my chef server.
publishsettings file exists in my .chef directory right next to knife.rb, all connections work and deploying a 2012r2 image works flawlessly.

Edit: Version of knife-azure is 1.4.0
Bootstrapped version of chef is 12.0.3 (default)

PS C:\Users\Galen\OneDrive\chef\galenemery> knife azure server create --azure-dns-name galen-2008r2 --azure-vm-size Medi
um -I a699494373c04fc0bc8f2bb1389d6106__Win2K8R2SP1-Datacenter-201412.01-en.us-127GB.vhd --azure-service-location 'North
 Central US' -x galen -P 'Chef123!' --bootstrap-protocol cloud-api -r 'fourthcoffee' --tcp-endpoints 80:80,3389:3389
.............
Waiting for virtual machine to reach status 'provisioning'............vm state 'provisioning' reached after 2.61 minutes
.
Waiting for virtual machine to reach status 'ready'..................vm state 'ready' reached after 4.17 minutes.
.
DNS Name: galen-2008r2.cloudapp.net
VM Name: galen-2008r2
Size: Medium
Azure Source Image: a699494373c04fc0bc8f2bb1389d6106__Win2K8R2SP1-Datacenter-201412.01-en.us-127GB.vhd
Azure Service Location: North Central US
Public Ip Address: 23.100.74.92
Private Ip Address: 100.78.6.54
WinRM Port: 5985
TCP Ports: [{"Name"=>"tcpport_3389_galen-2008r2", "Vip"=>"23.100.74.92", "PublicPort"=>"3389", "LocalPort"=>"3389"}, {"N
ame"=>"tcpport_80_galen-2008r2", "Vip"=>"23.100.74.92", "PublicPort"=>"80", "LocalPort"=>"80"}]
Environment: _default
Runlist: ["fourthcoffee"]

Waiting for Resource Extension to reach status 'wagent provisioning'....Resource extension state 'wagent provisioning' r
eached after 0.04 minutes.
Waiting for Resource Extension to reach status 'installing'....................Resource extension state 'installing' rea
ched after 2.23 minutes.
Waiting for Resource Extension to reach status 'provisioning'....................Resource extension state 'provisioning'
 reached after 2.21 minutes.
Waiting for Resource Extension to reach status 'ready'....................Resource extension state 'ready' reached after
 2.23 minutes.
.
DNS Name: galen-2008r2.cloudapp.net
VM Name: galen-2008r2
Size: Medium
Azure Source Image: a699494373c04fc0bc8f2bb1389d6106__Win2K8R2SP1-Datacenter-201412.01-en.us-127GB.vhd
Azure Service Location: North Central US
Public Ip Address: 23.100.74.92
Private Ip Address: 100.78.6.54
WinRM Port: 5985
TCP Ports: [{"Name"=>"tcpport_3389_galen-2008r2", "Vip"=>"23.100.74.92", "PublicPort"=>"3389", "LocalPort"=>"3389"}, {"N
ame"=>"tcpport_80_galen-2008r2", "Vip"=>"23.100.74.92", "PublicPort"=>"80", "LocalPort"=>"80"}]
Environment: _default
Runlist: ["fourthcoffee"]

PS C:\Users\Galen\OneDrive\chef\galenemery>
PS C:\Users\Galen\OneDrive\chef\galenemery> knife client list
54.187.33.204
Eros.chef.io
chef-endpoint
eros-validator
fourthcoffee05
galenemery-validator
hyperion-validator
web01.galenemery.com
PS C:\Users\Galen\OneDrive\chef\galenemery>

On the remote node, if I look at the c:\chef directory, I see everything I should need:

PS C:\Users\galen> ls C:\Chef


    Directory: C:\Chef


Mode                LastWriteTime     Length Name
----                -------------     ------ ----
d----         3/12/2015   1:22 AM            cache
-a---         3/12/2015   1:20 AM        940 client.rb
-a---         3/12/2015   1:20 AM         44 first-boot.json
-a---         3/12/2015   1:20 AM       1745 validation.pem


PS C:\Users\galen>

However, chef-client fails to register:

PS C:\Users\galen> chef-client
Starting Chef Client, version 12.0.3
Creating a new client identity for galen-2008r2 using the validator key.

================================================================================
Chef encountered an error attempting to create the client "galen-2008r2"
================================================================================

Chef Client failed. 0 resources updated in 37.40625 seconds
PS C:\Users\galen>

This is most likely due to some weird formatting happening with the validator.pem in that it doesn't appear clean, compared to the pem file on my workstation. The lines are not of equal length.

Replacing the validation.pem with a copy and paste from the one on my workstation resolves the issue and the system can bootstrap.

Bug: Nodes which have been created and joined domain do not get delete from Chef Server with knife azure delete --purge -y command.

I would speculate that's because the node name become FQDN when joined to a domain:
Example: azureServer401 will become azureServer401.microsoft.com

/cc: @msnahill for more info

Bug: specifying the full Azure URL in the knife.rb causes knife azure commands to fail.
Expectation: specifying the full URL should work.

Azure docs and config files use the full https URL
example from azureProfile.json:
"managementEndpointUrl": "https://management.core.windows.net"
example from myazure.publishsettings:
ServiceManagementUrl="https://management.core.windows.net"

I think the knife setting should accept both, the short URL as it has presently (to avoid breaking current users) and in the full https:/... format.

On knife-azure 1.4.0, neither --environment or --json-attributes seem to be propogated to my (Linux) machine, either at runtime on the node or back to the chef server subsequent to a successful run. Are these features working, and if so, are there any caveats on their usage? If not, is there another mechanism I can use to pass run-specific information to a machine?

thanks,

Scott.

Dear all,

I use 'knife azure' to manage my azure infrastructure. But I found 'knife azure server create' was a synchronous request, which means I must wait for one VM's creation completed to create a next VM.

Does 'knife azure server create' support asynchronous request? I want to send multiple 'knife azure server create' request at the same time to create VMs simultaneously.

Thanks,
Chao Jin

hi! I have not started using Chef yet, I'm trying to figure out how/where I can install. So I apologize if I am misunderstaning things.

From what I have read, all the other configuration management solutions, ie Ansible, Puppet, SaltStack, have a hard requirement that the 'leader' node, from which one gives orders, must be a Linux / Unix machine.

However, with this new build of knife-azure, which installs via Ruby gems, do I understand correctly that one will not need to use a 'chef server' in this configuration?

Instead, I will install knife-azure onto my Windows 8.1 machine, hook it up to my Azure credentials, and populate my chef recipes I wish for Azure, right?

Furthermore, my Win8.1 machine can tell Azure to create either Windows or Linux machines, right?

So from a windows machine, I could thus spawn off, monitor, and control Azure instances of any type of machine, including Docker-enabled Ubuntu 14.04's.

thank you for this clarification! All the downloads page only reference linux, that's why I have been confused.
AnneTheAgile

Refs;
1.[] ; ; Reason of windows-leader support is not mentioned; ; ; ; ; ; ; ; X.Chef over Puppet on Azure? - Server Fault
http://serverfault.com/questions/639856/chef-over-puppet-on-azure
2.[] ; ; ; ; ; ; ; ; ; ; X.chef server/puppet master installation on windows operating system - Stack Overflow
http://stackoverflow.com/questions/19380687/chef-server-puppet-master-installation-on-windows-operating-system
3.[] ; ; ; ; ; ; ; ; ; ; X.Running Chef recipes using Vagrant on Windows host: issue with SSH - Stack Overflow
http://stackoverflow.com/questions/11268472/running-chef-recipes-using-vagrant-on-windows-host-issue-with-ssh
4,[]Chef docs ; refs to Videos ; ; ; ; ; ; ; ; ; X.Windows Azure | Chef
https://www.chef.io/partners/microsoft/
5.[] ;Windows docs' imply I cannot use windows as the leader node ; ; ; ; ; ; ; ; ; X.About Chef and Azure Virtual Machines
http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-install-chef-client/
6.[] Free chef server-leader, Up to 25 Nodes (Hosting up to 5 Nodes) ; ; ; ; ; ; ; ; ; ; X.Chef | IT automation for speed and awesomeness | Chef
https://www.chef.io/chef/#plans-and-pricing
7.[] You need to have knife installed on your machine. Go to http://getchef.com to install it on your machine and define your local repository.;22 Apr 2014 ; ; ; ; ; ; ; ; ; X.Hands-free configuration of your VM in Microsoft Azure - Damien Caro's Blog - Site Home - TechNet Blogs
http://blogs.technet.com/b/dcaro/archive/2014/04/22/hands-free-configuration-of-your-vm-in-microsoft-azure.aspx
8.[] ;Lots of great windows tools, eg Registry, Path, Reboot, Printer, Font , Installer , Windows Feature, Batch, AutoRun ; ; ; ; ; ; ; ; ; X.opscode-cookbooks/windows
https://github.com/opscode-cookbooks/windows

Command:
knife azure server delete 'machine01' 'machine02' 'machine03' 'machine04' 'machine05' 'machine06' --purge -y

Bug:
Machine06 was not deleted from Azure. It didn't even get listed in the command's output of work it was performing.

/cc: @msnahill for more info

.ERROR: Virtual machine state 'provisioning' not reached after 20 minutes.
ERROR: Failed to create the server -- exception being rescued: Verify connectivity to Azure and subscription resource li
mit compliance (e.g. maximum CPU core limits) and try again.

I am unable to understand what this means.

Issue

using --secret-file /path/to/secret_file command-line argument does not properly transfer the secret file to servers.

Knife bootstrap via other modes works as expected.

Workaround

Setting knife[:secret_file] in knife.rb works as expected with knife azure plugin.

It might be this:

./lib/chef/knife/azure_server_create.rb:733:734

      def bootstrap_for_windows_node(server, fqdn, port)
    // snip
        bootstrap.config[:encrypted_data_bag_secret] = locate_config_value(:encrypted_data_bag_secret)
        bootstrap.config[:encrypted_data_bag_secret_file] = locate_config_value(:encrypted_data_bag_secret_file)

vs.

./lib/chef/knife/azure_server_create.rb:753:754

      def bootstrap_for_node(server,fqdn,port)
    // snip

        bootstrap.config[:secret] = locate_config_value(:secret)
        bootstrap.config[:secret_file] = locate_config_value(:secret_file)

Thanks,
Eric

When I attempt to use the azure_tcp_endpoints setting in my knife.rb to set a port, this endpoint never gets created on the VM in azure.

The command I am running is

~$ knife azure server create --azure-dns-name 'ramsey-11143'

Attached is the settings in my knife.rb and then the endpoint configuration after the knife bootstrap successfully completes.

I have this in my knife.rb

knife[:azure_subscription_id] = "blahblah"
knife[:azure_mgmt_cert] = "ManagementCertificate.pem"
knife[:azure_host_name] = "management-preview.core.windows-int.net"
knife[:service_location] = 'West US'
knife[:bootstrap_protocol] = 'winrm'
knife[:winrm_password] = 'asdasd!'
knife[:winrm_transport] = 'plaintext'
knife[:winrm_port] = '5985'
knife[:distro] = 'windows-chef-client-msi'
knife[:source_image]='w2k12Basic.vhd'
knife[:role_name]='soloist'
knife[:host_name]='soloist'
knife[:role_size]='Small'

I get error creating server

➜  chef-repo git:(master) ✗ knife azure server create -VV
DEBUG: Using configuration from /Users/nilesh/projects/chef-repo/.chef/knife.rb
INFO: validating...
INFO: creating...
INFO: Using the winrm protocol for bootstrapping
INFO: calling get storageservices
WARNING: Could not load IOV methods. Check your GSSAPI C library for an update
WARNING: Could not load AEAD methods. Check your GSSAPI C library for an update
INFO: calling get images
/Users/nilesh/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/knife-azure-1.1.0/lib/chef/knife/azure_base.rb:67:in `is_image_windows?': undefined method `os' for nil:NilClass (NoMethodError)
from /Users/nilesh/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/knife-azure-1.1.0/lib/chef/knife/azure_server_create.rb:423:in `create_server_def'
from /Users/nilesh/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/knife-azure-1.1.0/lib/chef/knife/azure_server_create.rb:280:in `run'
from /Users/nilesh/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/chef-10.16.2/lib/chef/knife.rb:408:in `run_with_pretty_exceptions'
from /Users/nilesh/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/chef-10.16.2/lib/chef/knife.rb:168:in `run'
from /Users/nilesh/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/chef-10.16.2/lib/chef/application/knife.rb:123:in `run'
from /Users/nilesh/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/chef-10.16.2/bin/knife:25:in `<top (required)>'
from /Users/nilesh/.rbenv/versions/1.9.3-p194/bin/knife:23:in `load'
from /Users/nilesh/.rbenv/versions/1.9.3-p194/bin/knife:23:in `<main>'

This is probably a configuration issue rather than a bug, but I've followed the docs as best I can and am having some trouble provisioning windows VMs, linux ones work fine.

I'm using 1.2.0, which I checked out and built a few days back (Thursday, I believe).

I created a windows VM in Azure, followed the docs on the knife-windows project to configure winrm (re: ran all of the commands that are shown there), ran sysprep on the machine, and captured an image from it.

Then I configure my knife.rb with my subscription and cert info (which works since I can provision linux VMs fine) and feed that image to the knife azure server create command as follows:

knife azure -VV server create --azure-service-location 'West US' --winrm-user myusername --winrm-password password123? --azure-source-image quickchef-windows2008 --azure-dns-name winChef1234 --azure-vm-size Small --azure-storage-account myvmstorage --template-file windows-template.erb

The VM gets created with the WinRM endpoint and then fails to connect with a 401. I'm guessing it's a azure/winrm/sysprep issue, but there isn't really documentation that I've been able to find discussing it. Here is the long output from the command that I've run https://gist.github.com/sethvargo/5980750

Thanks!

(I moved the stacktrace to a gist to improve readability - @sethvargo)

LoadBalancedEndpoint, Subnet, AvailabilitySet features are required when creating azure VM.

For WinRM feature, although it's fine with a pre-captured image with WinRM enabled, it's nice to enable this feature to directly use standard Windows image.

In my knife.rb , for my run_list i gave getting-started recipe but it seems to be ignored or overwritten for command knife azure server create

knife[:run_list]='recipe[getting-started]'

Below is how my knife.rb looks like and

current_dir = File.dirname(FILE)
log_level :info
log_location STDOUT
node_name "arunapadala"
client_key "#{current_dir}/arunapadala.pem"
validation_client_name "aruna1-validator"
validation_key "#{current_dir}/aruna1-validator.pem"
chef_server_url "https://api.opscode.com/organizations/aruna1"
cache_type 'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path ["#{current_dir}/../cookbooks"]
knife[:azure_subscription_id] = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
knife[:azure_mgmt_cert] = "publishsettings_decoded.pem"
knife[:azure_host_name] = "management.core.windows.net"
knife[:service_location] = 'West US'
knife[:hosted_service_name]='chef4sur'
knife[:role_name]='chef4sur'
knife[:host_name]='chef4sur'
knife[:distro]='ubuntu12.04-gems'
knife[:ssh_user]='ubuntu'
knife[:ssh_password]='a1B2C3D4@'
knife[:storage_account]='sureshtest'
knife[:tcp_endpoints]='66'
knife[:udp_endpoints]='77,88,99'
knife[:run_list]='recipe[getting-started]'
knife[:source_image]='CANONICAL__Canonical-Ubuntu-12.04-amd64-server-20120924-en-us-30GB.vhd'
knife[:role_size]='Medium'

Some sysprepped images never reach the "ready" state in Azure, and knife-azure gives up after waiting for a maximum timeout. The Azure portal will show the VM in a "provisioning" state rather than "running" indefinitely. However, the image is actually ready and can be accessed over WinRM. This does not occur with Azure's stock images. Here's some example output from such a failed create + bootstrap:

.............
Waiting for virtual machine to reach status 'provisioning'..........`..vm state 'provisioning' reached after 2.64 minutes.
Waiting for virtual machine to reach status 'ready'............................................................ERROR: Virtual machine state 'ready' not reached after 15 minutes.
ERROR: Failed to create the server -- exception being rescued: Verify connectivity to Azure and subscription resource limit compliance (e.g. maximum CPU core limits) and try again.

Hi

I am able to create Windows Azure Persistent VM using knife azure plugin(knife azure server create) ,
i also observed that chef-client is configured automatically.

However facing issue when using knife ssh , used below commands
knife ssh name:sureshrole02 -x ubantu -P a1B2C3D4@ "sudo chef-client"
or
knife ssh name:sureshrole02 -x ubantu -P a1B2C3D4@ "sudo chef-client" -a ipaddress

Getting below error
WARNING: Failed to connect to node[sureshrole02] -- SocketError: getaddrinfo: Temporary failure in name resolution

However please note that i am able to login using plain ssh(ssh username@public IP address)

current_dir = File.dirname(FILE)
log_level :info
log_location STDOUT
node_name "arunapadala"
client_key "#{current_dir}/arunapadala.pem"
validation_client_name "aruna1-validator"
validation_key "#{current_dir}/aruna1-validator.pem"
chef_server_url "https://api.opscode.com/organizations/aruna1"
cache_type 'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path ["#{current_dir}/../cookbooks"]
knife[:azure_subscription_id] = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
knife[:azure_mgmt_cert] = "publishsettings_decoded.pem"
knife[:azure_host_name] = "management.core.windows.net"
knife[:service_location] = 'West US'
knife[:hosted_service_name]='sureshhs2'
knife[:role_name]='sureshrole02'
knife[:host_name]='sureshhost02'
knife[:distro]='ubuntu12.04-gems'
knife[:ssh_user]='ubantu'
knife[:ssh_password]='a1B2C3D4@'
knife[:storage_account]='sureshtest'
knife[:tcp_endpoints]='66'
knife[:udp_endpoints]='77,88,99'

knife[:source_image]='CANONICAL__Canonical-Ubuntu-12.04-amd64-server-20120924-en-us-30GB.vhd'
knife[:role_size]='Medium'

knife azure server create -N MyNewNode --azure-vm-size Medium --I 8fcc3d_Ubuntu-12_04-amd64-30GB -m 'West US' --ssh-user myuser --identity-file ~/.ssh/myprivatekey_rsa

should be

knife azure server create -N MyNewNode --azure-vm-size Medium -I 8fcc3d_Ubuntu-12_04-amd64-30GB -m 'West US' --ssh-user myuser --identity-file ~/.ssh/myprivatekey_rsa

Using --I causes

ERROR: You did not provide a valid 'Azure Source Image' value. Please set knife[:azure_source_image] in your knife.rb or pass as an option.`

I'm running the following command:

knife azure server create --azure-dns-name 'knifeth4' --azure-source-image "a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201408.01-en.us-127GB.vhd" --azure-service-location "West US" -P "Pizza1234" -z Small -x "thiago" --tcp-endpoints '3389:3389,5985:5985' -r "role[learn_chef_iis],role[nopcommerce]" -VV

It configure endpoints and also create vm. But when it tries to execute run list, it exits and stop the execution:

WinRM Port: 5985
Environment: _default
Runlist: ["role[iis]"]
INFO: calling get hostedservices/az-933158e2aac5/deploymentslots/Production sync
hronously
.
DNS Name: az-933158e2aac5.cloudapp.net
VM Name: az-933158e2aac5
Size: Small
Azure Source Image: a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datace
nter-201408.01-en.us-127GB.vhd
Azure Service Location: West US
Public Ip Address: 104.40.0.138
Private Ip Address: 100.74.192.93
WinRM Port: 5985
Environment: _default
Runlist: ["role[iis]"]
Waiting for winrm on 104.40.0.138:5985
Waiting for remote response before bootstrap.DEBUG: Adding 104.40.0.138
DEBUG: :session => :init
DEBUG: :relay_to_servers => echo . & echo Response received.
DEBUG: :relayed => 104.40.0.138
DEBUG: 104.40.0.138 => :run_command
ERROR: No response received from remote node after 0.01 minutes, giving up.
ERROR: RuntimeError: Command execution failed.

I'm stuck with this error. Am I missing something?

I'm a bit puzzled about the parameter --identity-file.

Can you please explain to me, why I have to specify the private rsa key? The private key is supposed to stay on my local machine and only the public key should be transferred to the remote machine.

I really do not want my private key to be uploaded somewhere. Something is off here.

When deploying with the following command, using knife-azure 1.6.0rc0 I get the error in the subject.
If I actively change the header value in the gem to request["x-ms-version"] = "2015-04-01", it works fine.

Could you please bump it to 2015-04-01?

knife azure server create --azure-dns-name "future01" --azure-vm-size Standard_DS2 --azure-service-location 'North Europe' --winrm-user kekkonen1 --winrm-password 'lolophone' --run-list "nightly" --environment "azure-nightly" -VV

The prompt for knife azure server delete should allow upper and lower case [y|Y] or allow the literal string being requested.

Do you really want to delete this server? (Y/N) Y I have no idea what to do with 'Y' Just say Y or N, please. Do you really want to delete this server? (Y/N) y

Bug:
When --node-ssl-verify-mode none is specified no changes are made to client.rb

Expectation:
ssl_verify_mode :verify_none is rendered to the client.rb

Actual:
no changes are made to the file

Notes:

• --no-node-verify-api-cert does place additional text in the client.rb (does not resolve auth with custom trust auth certificates)
• a custom gem is available for testing (verify now) - adamedx/node_ssl_verify

There seems to be a rdoc conflict with chefdk. When installing knife-azure with the chef gem install command, the following occurs with any knife commands:

/opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/specification.rb:2064:in `raise_if_conflicts': Unable to activate knife-azure-1.2.2, because rdoc-4.1.0 conflicts with rdoc (~> 3.12) (Gem::LoadError)

Given the following setup:

• Provision VM vm1 into cloud service cs1 using knife-azure.
• Configure vm1 using the management portal. Add a stand alone endpoint on Port 80 and make that endpoint part of a load balanced set, lbs1.

I expect to be able to provision a new VM, vm2 in cs1 using lbs1 but I can not.
Even better, I want to be able to have lbs1 created when I create vm1 and then be able to add more VMs to lbs1 later.

Thanks to knife-azure that I can use knife to manage azure servers. But 'knife azure server create' command can only create one vm at a time. How can I create multiple VMs at the same time under one 'knife azure server create' command? Or is there any other solutions?

Thanks,
Chao Jin

Both the --distro and --azure-dns-name options have a short -d option. The --azure-dns-name option should not have the -d option. Additionally, we should have a test that verifies that there are no duplicate short or long options that would have failed in this case.

We should allow an option in knife-azure to allow the script extension to run before bootstrap when winrm is used (not cloud-api). The user can then specify a powershell script as a string that can be run by the VM script extension to do things like set the winrm timeout, which is useful for Win2k8r2 systems that have a bad winrm config.

1. Configured Chef-workstation on windows machine
2. Installed knife azure gem by running the command "bundle install" , followed by "rake install"
3. Uploaded all the cookbooks from https://github.com/opscode/java-quick-start.git to hosted chef
4. Executed server create command:

    $ knife azure server create --hosted-service-name myHostedService0430 --role-name role5 --run-list 'recipe[java]’

As expected it created a Windows Azure Ubantu VM but looks like run-list info is not getting passed to the VM.

its failing with error "NoMethodError: undefined method `run_list' for nil:NilClass"

I am sure that thus used to work earlier , not sure what got broken recently

Below is how my knife.rb looks like:

current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "snallami2"
client_key               "#{current_dir}/snallami2.pem"
validation_client_name   "snallami2-validator"
validation_key           "#{current_dir}/snallami2-validator.pem"
chef_server_url          "https://api.opscode.com/organizations/snallami2"
cache_type               'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{current_dir}/../cookbooks"]
knife[:azure_subscription_id] = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
knife[:azure_mgmt_cert] = "managementCertificate.pem"
knife[:azure_host_name] = "management.core.windows.net"
knife[:service_location] = 'West US'
knife[:hosted_service_name]='chef3sur'
knife[:role_name]='chef3sur'
knife[:host_name]='chef3sur'
knife[:distro]='ubuntu12.04-gems'
knife[:ssh_user]='ubuntu'
knife[:ssh_password]='aBCD.1234'
knife[:storage_account]='sureshtest'
knife[:tcp_endpoints]='66,80,8080,22002,3306'
knife[:udp_endpoints]='77,88,99'
knife[:source_image]='b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-12_04_1-LTS-amd64-server-20121218-en-us-30GB'
knife[:role_size]='Medium'

Has anyone tested this to see if it's compatible with the on-prem Windows Azure Pack (WAP)?

Bug: below command is not able to join newly created Azure server to Domain.

Command:
knife azure server create --azure-dns-name myserver --azure-vm-size Medium -I a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-20150916-en.us-127GB.vhd --azure-service-location 'Central US' --winrm-user myuser --winrm-password 'cleartextpassword' --bootstrap-protocol cloud-api --azure-network-name 'vnet_express_route' --azure-subnet-name 'Tenant Subnet' --no-node-verify-api-cert --node-ssl-verify-mode none --azure-vm-startup-timeout 15 --azure-domain-name 'ExpressRoute.domain.example.com' --azure-domain-passwd 'AnotherClearTextPassword' --azure-domain-user 'ExpressRoute\domainaccount' --azure-domain-ou-dn 'OU=SubFolder,OU=Folder,DC=ExpressRoute,DC=domain,DC=example,DC=com'

Notes:
If the command is executed with out the --azure-domain-ou-dn flag, the error is the same.

Error:
..........FATAL: Unable to create role:BadRequest : Invalid domain join information specified. Ensure that the JoinDomain setting is specified, and that either the Credentials or Provisioning setting is specified. See the Windows Automated Installation Kit documentation for more information. ERROR: Failed to create the server -- exception being rescued: exception class/object expected WARNING: Cleaning up resources... ..WARNING: Deleted created DNS: myserver
There is no stacktrace

Running into this inconsistently - some kind of race condition? Using this command...

knife azure server create - --azure-vm-size Small --azure
-dns-name 'chefsrv90' --azure-vm-name 'chefsrv90' --azure-affinity-group chefnet
-r "role[webserver]" --azure-network-name chefnet --tcp-endpoints 80:80,8080:80
80

provisions, then once its up during client install:

to file /tmp/install.sh.1434/chef_11.12.4-1_amd64.deb
trying wget...
Comparing checksum with sha256sum...
Installing Chef 11.12.4
installing with dpkg...
pkg: error: dpkg status database is locked by another process
Installation failed
Please file a bug report at http://tickets.opscode.com
Project: Chef
Component: Packages
Label: Omnibus
Version: 11.12.4
Please detail your operating system type, version and any other r
elevant details
Starting first Chef Client run...
bash: line 90: chef-client: command not found

When using winrm or ssh, knife will automatically copy over ssl certs to the node being bootstrapped. This should also be done with cloud-api bootstrap.
Support will be needed in both the azure_chef_extension and in knife-azure.

re: https://chefio.atlassian.net/browse/CLOGENG-134

@msnahill

How can I have knife-azure provision custom images I have stored in azure?

Hi,

Is there any support for Azure China? I suspect it has some minor differences - Has anyone else tested it? I notice that it isn't in the region list in the documentation (although these are only examples..).

Thanks,
Aiman

I'm trying to create a VNET with the command
knife azure vnet create -n 'ChefDemoVNET' -a 'ChefDemoAG' --azure-address-space '10.0.0.0/24'
where ChefDemoAG is an existing ag.

Error out with the message:
.ERROR: knife encountered an unexpected error
This may be a bug in the 'azure vnet create' knife command or plugin
Please collect the output of this command with the -VV option before filing a
bug report.
Exception: NoMethodError: undefined method `add_next_sibling' for nil:NilClass

Hello,

when provisioning a new VM and joining it to an existing cloud service I get following error (with -VV):

.DEBUG: <?xml version="1.0"?>
<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <Code>InvalidXmlRequest</Code>
  <Message>The request body's XML was invalid or not correctly specified.</Message>
</Error>

DEBUG: InvalidXmlRequest : The request body's XML was invalid or not correctly specified.
DEBUG: <?xml version="1.0"?>
<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <Code>InvalidXmlRequest</Code>
  <Message>The request body's XML was invalid or not correctly specified.</Message>
</Error>

FATAL: Unable to create role:InvalidXmlRequest : The request body's XML was invalid or not correctly specified.
ERROR: Failed to create the server -- exception being rescued: exception class/object expected
DEBUG: TypeError: exception class/object expected
/home/vagrant/.chefdk/gem/ruby/2.1.0/gems/knife-azure-1.5.2/lib/azure/deploy.rb:103:in `raise'
/home/vagrant/.chefdk/gem/ruby/2.1.0/gems/knife-azure-1.5.2/lib/azure/deploy.rb:103:in `create'
/home/vagrant/.chefdk/gem/ruby/2.1.0/gems/knife-azure-1.5.2/lib/chef/knife/azure_server_create.rb:631:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:405:in `block in run_with_pretty_exceptions'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/local_mode.rb:44:in `with_server_connectivity'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:404:in `run_with_pretty_exceptions'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:203:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application/knife.rb:142:in `run'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/bin/knife:25:in `<top (required)>'
/opt/chefdk/bin/knife:54:in `load'
/opt/chefdk/bin/knife:54:in `<main>'

It was happening on my local machine, did a clean install on a vagrant Ubuntu 14.04 LTS box with the same result, as you can see above.

I was able to fix it by installing the 1.4.0 version, where the same server create commands works.

vagrant@vagrant:~$ gem uninstall knife-azure
Successfully uninstalled knife-azure-1.5.2

vagrant@vagrant:~$ gem install knife-azure -v 1.4.0 
Fetching: knife-azure-1.4.0.gem (100%)
Successfully installed knife-azure-1.4.0

Maybe I am just not aware of some configuration changes in the latest version and this is an expected behaviour.

"Knife azure server create" is giving error when using winrm enabled images

Below is the error message

Waiting for winrm on 168.62.204.89:5985......ERROR: knife encountered an unexpected error
This may be a bug in the 'azure server create' knife command or plugin
Please collect the output of this command with the -VV option before filing a bug report.
Exception: NoMethodError: undefined method `id' for #Azure::Role:0x000000041af820.

After the VM is provisioned i tried manually below command and it worked
knife bootstrap windows winrm chef15sur.cloudapp.net --winrm-user Administrator -P a1B2C3D4@

My Knife.rb

current_dir = File.dirname(FILE)
log_level :info
log_location STDOUT
node_name "arunapadala"
client_key "#{current_dir}/arunapadala.pem"
validation_client_name "aruna1-validator"
validation_key "#{current_dir}/aruna1-validator.pem"
chef_server_url "https://api.opscode.com/organizations/aruna1"
cache_type 'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path ["#{current_dir}/../cookbooks"]
knife[:azure_subscription_id] = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
knife[:azure_mgmt_cert] = "publishsettings_decoded.pem"
knife[:azure_host_name] = "management.core.windows.net"
knife[:service_location] = 'West US'
knife[:hosted_service_name]='chef11sur'
knife[:role_name]='chef11sur'
knife[:host_name]='chef11sur'
knife[:storage_account]='sureshtest'
knife[:tcp_endpoints]='66,80'
knife[:udp_endpoints]='77,88,99'
knife[:run_list]='recipe[getting-started]'
knife[:source_image]='win2k8winrm2'
knife[:role_size]='Medium'
knife[:bootstrap_protocol]='winrm'
knife[:winrm_password]='a1B2C3D4@'
knife[:winrm_transport]='plaintext'
knife[:winrm_port]='5985'
knife[:distro]='windows-chef-client-msi'

https://msdn.microsoft.com/en-us/library/azure/ee460804.aspx

We can get the right format and value of Azure REST api (Get-Deployment) if created the VM by Azure.
By Powershell to get Ge-Deployment :

By Azure JAVA SDK to get Get-Deployment :

However, we found it failed with wrong format and value if created by knife-azure.
By Powershell to get Ge-Deployment :

By Azure JAVA SDK to get Ge-Deployment :

The wrong format of Get-Deployment create by knife-azure :
chris-test1Productionfed1879427af455898376c545a779d5fSuspendedY2hyaXMtOTUyLTI=http://chris-test1.cloudapp.net/PFNlcnZpY2VDb25maWd1cmF0aW9uIHhtbG5zOnhzZD0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEiIHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiIHhtbG5zPSJodHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL1NlcnZpY2VIb3N0aW5nLzIwMDgvMTAvU2VydmljZUNvbmZpZ3VyYXRpb24iPg0KICA8Um9sZSBuYW1lPSJjaHJpcy05NTItMiI+DQogICAgPEluc3RhbmNlcyBjb3VudD0iMSIgLz4NCiAgPC9Sb2xlPg0KPC9TZXJ2aWNlQ29uZmlndXJhdGlvbj4=chris-test1chris-test1StoppedDeallocatedStopped6904E4DFCC84CEBEFBB265E532BB25112987A9CC1Rolei:type="PersistentVMRole"chris-test1PersistentVMRoleNetworkConfiguration5986PowerShell5986tcpfalse3389Remote Desktop3389tcpfalseReadWritechris-test1-chris-test1-0-201505050830580485https://portalvhdsdqnt01flqwh9p.blob.core.windows.net/vhds/chris-test1-chris-test1-2015-05-05.vhda699494373c04fc0bc8f2bb1389d6106__Win2K8R2SP1-Datacenter-201504.01-en.us-127GB.vhdWindowsStandard_D1EAEA705B4F65B8A286B1CE95CF9DD58AA77FF553truefalsefalse2015-05-05T08:30:53Z2015-05-05T08:53:16Zchris-test1.g6.internal.cloudapp.net

Would there be any functionality to create Azure resources such as VM using Azure ARM JSON templates?

https://github.com/Azure/azure-quickstart-templates

When I attempt to install the knife-azure gem on two different computers under osx I get the following error:

ERROR: Error installing knife-azure:
knife-azure requires knife-azure (>= 0)

I have tried with and without sudo.

It looks like we construct a client.rb file to ship via cloud-api (and possibly bootstrap templates as well if we're doing WinRM) - it'd be great to have support for specifying --[no-]node-verify-api-cert in order to slip in an option in client.rb to not verify the Chef Server SSL certificate in the case of using self-signed certificates.

Hi all - not sure why this is happening, here's my output:

azureuser@:~$ knife azure server create -N MyNewNodet --azure-vm-size Small --azure-source-image "0b11de9248dd4d87b18621318e037d37__RightImage-Ubuntu-12.04-x64-v5.8.8.8" -m 'West US' --ssh-us
er azureuser --identity-file ~/myPrivateKey.key
............
Waiting for virtual machine to reach status 'provisioning'............vm state 'provisioning' reached after 2.58 minutes.
.
DNS Name: ************
VM Name: *******************
Size: Small
Azure Source Image: 0b11de9248dd4d87b18621318e037d37__RightImage-Ubuntu-12.04-x64-v5.8.8.8
Azure Service Location: West US
Public Ip Address: ****************
Private Ip Address: ********************
SSH Port: 22
Environment: _default

Waiting for sshd on *************:22done

Connecting to *****************
**************** [sudo] password for azureuser: Sorry, try again.
**************** [sudo] password for azureuser:
**************** Sorry, try again.
**************** [sudo] password for azureuser:
**************** Sorry, try again.
**************** sudo: 3 incorrect password attempts

If I ssh azureuser@ip and accept the thumbprint I connect just fine without a password - is there a specific reason why the knife bootstrap is trying a password?

On my ubuntu 12.04LTS machine x86_64 , installed ruby and chef using omnibus installer

commands
$ curl -L https://www.opscode.com/chef/install.sh | sudo bash
$ echo 'export PATH="/opt/chef/embedded/bin:$PATH"' >> ~/.bash_profile && source ~/.bash_profile

Then cloned knife azure plugin and tried bundle install , it got stuck for ever.

When trying to bootstrap a base linux box I found a bug where the port isn't properly getting passed into the generated xml when creating the ssh endpoint, causing azure to complain about not liking the generated xml. The problem appears to be that params[:port] isn't available when we attempt to inject it here

<?xml version="1.0"?>
<Deployment xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <Name>az-0682f646415e</Name>
  <DeploymentSlot>Production</DeploymentSlot>
  <Label>YXotMDY4MmY2NDY0MTVl</Label>
  <RoleList>
    <Role i:type="PersistentVMRole">
      <RoleName>az-0682f646415e</RoleName>
      <OsVersion i:nil="true"/>
      <RoleType>PersistentVMRole</RoleType>
      <ConfigurationSets>
  <ConfigurationSet i:type="LinuxProvisioningConfigurationSet">
    <ConfigurationSetType>LinuxProvisioningConfiguration</ConfigurationSetType>
    <HostName>az-0682f646415e</HostName>
    <UserName>andre</UserName>
    <UserPassword>****</UserPassword>
    <DisableSshPasswordAuthentication>false</DisableSshPasswordAuthentication>
  </ConfigurationSet>
  <ConfigurationSet i:type="NetworkConfigurationSet">
    <ConfigurationSetType>NetworkConfiguration</ConfigurationSetType>
    <InputEndpoints>
      <InputEndpoint>
        <LocalPort>22</LocalPort>
        <Name>SSH</Name>
        <Port/>
        <Protocol>TCP</Protocol>
      </InputEndpoint>
    </InputEndpoints>
  </ConfigurationSet>
</ConfigurationSets>
      <VMImageName>ChefServer_BYOL_1446255674_2015-10-30_18-41</VMImageName>
      <Label>YXotMDY4MmY2NDY0MTVl</Label>
      <RoleSize>Small</RoleSize>
    </Role>
  </RoleList>
</Deployment>

I'm trying to create a vm with knife-azure using:

knife azure server create -VV --azure-dns-name 'example-vm-name' --azure-vm-size 'Basic_A2' --azure-service-location 'Japan West' --azure-source-image 'example-image-name'

but consistently getting:

A ProvisioningConfigurationSet was not found in the ConfigurationSet collection of the virtual machine with name <name>. When creating a virtual machine from an image, ProvisioningConfigurationSet must be specified.

I have the ssh_user, identity_file and publish_settings file set correctly in knife.rb, but the xml that gets generated doesn't include ProvisioningConfigurationSet. Is this a new or known issue?

using:
Chef: 12.3.0
knife-azure: 1.4.0

Have been trying to create azure vm using knife using a hosted chef server but have been failing to do so . It waits for winrm to respond and then finally kills itself.

Can someone please provide me help to get me unblocked its very urgent for me to move ahead from here.

Any help will be highly appreciated.
Thanks in advance.

PS C:\chef-repo> knife azure server create --azure-connect-to-existing-dns --azure-dns-name chefnode --azure-vm-name 'node5' --azure-service-location 'East Asia' --azure-source-image 'bd507d3a70934695bc2128e3e5a255ba__RightImage-Windows-2008R2-SP1-x6

4-iis75-v5.8.8' --winrm-user 'node' --winrm-password 'node12!@' --bootstrap-protocol winrm --distro 'windows-chef-client-msi'

..........

Waiting for virtual machine to reach status 'provisioning'............vm state 'provisioning' reached after 2.85 minutes.

Waiting for virtual machine to reach status 'ready'..........................vm state 'ready' reached after 6.76 minutes.

.

DNS Name: chefnode.cloudapp.net

VM Name: node5

Size: Small

Azure Source Image: bd507d3a70934695bc2128e3e5a255ba__RightImage-Windows-2008R2-SP1-x64-iis75-v5.8.8

Azure Service Location: East Asia

Public Ip Address: 168.63.212.145

Private Ip Address: 100.80.52.52

WinRM Port: 47077

Environment: _default

Waiting for winrm on 168.63.212.145:47077

Waiting for remote response before bootstrap.WARNING: Unsupported version of httpclient. The supported major version of library is httpclient version 2.4. This code path monkey patches few methods in httpclient to support additional features. This co

uld possibly work, but it is advised to extensively test your version. If you are aware of the impact of using 2.5.0, this warning can be disabled by setting USE_HTTPCLIENT_MAJOR to the major version 2.5.

ERROR: No response received from remote node after 0.01 minutes, giving up.

ERROR: RuntimeError: Command execution failed.

PS C:\chef-repo>

Thanks,
Pravin

I have a need to upload a new service package using the Azure Create Deployment API. Is there a way to do this currently? If not should I implement the functionality?

i.e. http://msdn.microsoft.com/en-us/library/ee460813.aspx