cloudfoundry-attic / bosh_vcloud_cpi Goto Github PK
View Code? Open in Web Editor NEWvCloud Cloud Provider Interface (CPI) for BOSH
Home Page: http://bosh.io/docs/vcloud-cpi.html
License: Apache License 2.0
vCloud Cloud Provider Interface (CPI) for BOSH
Home Page: http://bosh.io/docs/vcloud-cpi.html
License: Apache License 2.0
We seem to be having some trouble deploying microbosh to vcloud using the bosh_vcloud_cpi. It seems to be related to some sort of version negotiation issue.
At the top of the bosh_micro_deploy.log:
W, [2014-11-19T10:06:34.980812 #82186] [create_stemcell(/var/folders/4n/qs1rjbmd1c5gfb78m3_06j6r0000gn/T/d20141119-82186-1fxkf3n/sc-20141119-82186-15e256n/image, _)] WARN -- :
Caught exception when retrieving login url:
uninitialized constant VCloudSdk::Xml::SupportedVersions::VCloudClient"
Default to /api/sessions
Then later, when microbosh tried to create a VM (specifically setting the metadata), it explodes with the following:
/mnt/ci-tmp/ruby/1.9.1/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!': 405 Method Not Allowed (RestClient::MethodNotAllowed)
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/vcd_client.rb:235:in `block (2 levels) in send_request'
from /mnt/ci-tmp/ruby/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:228:in `call'
from /mnt/ci-tmp/ruby/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:228:in `process_result'
from /mnt/ci-tmp/ruby/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:178:in `block in transmit'
from /Users/pivotal/.rubies/ruby-1.9.3-p547/lib/ruby/1.9.1/net/http.rb:746:in `start'
from /mnt/ci-tmp/ruby/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:172:in `transmit'
from /mnt/ci-tmp/ruby/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute'
from /mnt/ci-tmp/ruby/1.9.1/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/vcd_client.rb:233:in `block in send_request'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/vcd_client.rb:271:in `block in retry_for_network_issue'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/vcd_client.rb:269:in `loop'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/vcd_client.rb:269:in `retry_for_network_issue'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/vcd_client.rb:231:in `send_request'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/vcd_client.rb:116:in `invoke'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/vcd_client.rb:126:in `invoke_and_wait'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/steps/save_agent_env.rb:25:in `perform'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/steps.rb:37:in `next'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/cloud.rb:314:in `save_agent_env'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/cloud.rb:130:in `block in create_vm'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/steps.rb:44:in `call'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/steps.rb:44:in `block in perform'
from /Users/pivotal/workspace/bosh/bosh_common/lib/common/thread_formatter.rb:49:in `with_thread_name'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/steps.rb:41:in `perform'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/steps.rb:58:in `perform'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/cloud.rb:287:in `steps'
from /mnt/ci-tmp/ruby/1.9.1/gems/bosh_vcloud_cpi-0.7.1/lib/cloud/vcloud/cloud.rb:50:in `create_vm'
from /Users/pivotal/workspace/bosh/bosh_cli_plugin_micro/lib/bosh/deployer/instance_manager.rb:243:in `create_vm'
from /Users/pivotal/workspace/bosh/bosh_cli_plugin_micro/lib/bosh/deployer/instance_manager.rb:123:in `block in create'
from /Users/pivotal/workspace/bosh/bosh_cli_plugin_micro/lib/bosh/deployer/instance_manager.rb:85:in `step'
from /Users/pivotal/workspace/bosh/bosh_cli_plugin_micro/lib/bosh/deployer/instance_manager.rb:122:in `create'
from /Users/pivotal/workspace/bosh/bosh_cli_plugin_micro/lib/bosh/deployer/instance_manager.rb:98:in `block in create_deployment'
from /Users/pivotal/workspace/bosh/bosh_cli_plugin_micro/lib/bosh/deployer/instance_manager.rb:92:in `with_lifecycle'
from /Users/pivotal/workspace/bosh/bosh_cli_plugin_micro/lib/bosh/deployer/instance_manager.rb:98:in `create_deployment'
from /Users/pivotal/workspace/bosh/bosh_cli_plugin_micro/lib/bosh/cli/commands/micro.rb:179:in `perform'
from /Users/pivotal/workspace/bosh/bosh_cli/lib/cli/command_handler.rb:57:in `run'
from /Users/pivotal/workspace/bosh/bosh_cli/lib/cli/runner.rb:56:in `run'
from /Users/pivotal/workspace/bosh/bosh_cli/lib/cli/runner.rb:16:in `run'
from /Users/pivotal/workspace/bosh/bosh_cli/bin/bosh:7:in `<top (required)>'
from /mnt/ci-tmp/ruby/1.9.1/bin/bosh:23:in `load'
from /mnt/ci-tmp/ruby/1.9.1/bin/bosh:23:in `<main>'
Here's the related log snipit (we had to enable DEBUG... the default log level INFO doesn't provide enough details):
D, [2014-11-19T10:10:39.523309 #82186] [create_vm(bm-2641f508-3191-4b44-9b1c-af5a79c23254, urn:vcloud:catalogitem:a2744507-574e-461b-89e9-2a1471cf2c52, {"ram"=>2048, "disk"=>8192, "cpu"=>1}, ...)] DEBUG -- : REST REQ PUT https://*****-vcd.vchs.vmware.com:443/api/vApp/vm-4fff38ab-829d-414c-b4e6-13b4aca757d8/metadata/ {:Accept=>"application/*+xml;version=5.1", :content_type=>"application/vnd.vmware.vcloud.metadata.value+xml", :x_vcloud_authorization=>"**************"} {"vcloud-token"=>"**************", "Path"=>"%2F"} <MetadataValue xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" type="application/vnd.vmware.vcloud.metadata.value+xml">
<TypedValue xsi:type="MetadataStringValue">
<Value>{"vm":{"name":"bm-2641f508-3191-4b44-9b1c-af5a79c23254","id":"urn:vcloud:vm:4fff38ab-829d-414c-b4e6-13b4aca757d8"},"agent_id":"bm-2641f508-3191-4b44-9b1c-af5a79c23254","disks":{"system":"0","ephemeral":"1","persistent":{}},"networks":{"bosh":{"cloud_properties":{"name":"VDC-BOSH"},"netmask":"255.255.255.0","gateway":"192.168.112.1","ip":"192.168.112.12","dns":["192.168.112.1"],"type":null,"default":["dns","gateway"],"mac":"00:50:56:0d:4b:bd"}},"env":{"bosh":{"password":null},"vapp":null},"ntp":[null],"blobstore":{"provider":"local","options":{"blobstore_path":"/var/vcap/micro_bosh/data/cache"}},"mbus":"https://vcap:**********@0.0.0.0:6868"}</Value>
</TypedValue>
</MetadataValue>
D, [2014-11-19T10:10:39.691651 #82186] [create_vm(bm-2641f508-3191-4b44-9b1c-af5a79c23254, urn:vcloud:catalogitem:a2744507-574e-461b-89e9-2a1471cf2c52, {"ram"=>2048, "disk"=>8192, "cpu"=>1}, ...)] DEBUG -- : REST RES 405 {:date=>"Wed, 19 Nov 2014 18:10:40 GMT", :x_vmware_vcloud_request_id=>"******************", :vary=>"Accept-Encoding", :x_vmware_vcloud_request_execution_time=>"7", :content_type=>"application/vnd.vmware.vcloud.error+xml;version=5.1", :content_length=>"341"} <?xml version="1.0" encoding="UTF-8"?>
<Error xmlns="http://www.vmware.com/vcloud/v1.5" majorErrorCode="405" message="HTTP method not allowed" minorErrorCode="METHOD_NOT_ALLOWED" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://10.6.54.3/api/v1.5/schema/master.xsd"></Error>
It appears to have defaulted to trying to use the v1.5 API, but it's using the PUT method instead of POST. I assume this means both version negotiation and v1.5 API support are broken.
I can supply the http://www.vmware.com/vcloud/versions response if necessary, but I think this is just a code issue.
I expected to vcloud CPI to support multi-homed VM.
https://bosh.io/docs/networks.html#multi-homed
https://bosh.io/docs/networks.html#cpi-limitations
My tests where unsuccessfull. Bosh Director gives 2 IP adress (ok on bosh vms), but the stemcell fails to configure the second NIC.
Having a look at the file /var/vcap/job/settings.json, I see that the 2 nics are set on the same MAC (whereas i have 2 distincts generated MACs, confirmed in vcloud portal).
For me its a blocker. I would be glad if you had any workaround.
Hey,
We would like to configure our microbosh to provision vms for our delployments with the Gold storage policy.
media_storage_profile:
does not cover this, and we would like to set the storage policy across a vapp.
We have tried setting vapp_storage_profile: Gold
in both
properties:
vcd:
entities:
vapp_storage_profile: Gold
and
plugin: vcloud
vcds:
- entities:
vapp_storage_profile: Gold
in our microbosh deployment manifest, but neither has worked.
On inspecting the cpi.json file in /var/vcap/jobs/cpi/config
we saw that this property does not exist.
Is there support for this? Or are we setting this property in the wrong section of our microbosh deployment manifest.
It is not clear whether the new functionality slowed down VM provisioning - it could simply be a busy cloud provider - however the "creating bound missing VMs" step failed with timeouts several times before succeeding, and was very slow. This doesn't usually happen.
It was mentioned on the mailing group discussion that "It is possible that VM provisioning is slightly slower since we are now recomposing a single vapp multiple times to create each of the VMs." Is there anything that could be done to speed up the process?
When a bosh deployment fails because of a failed vm instanciation, bosh then asks to clean up the failed vms by invoking the cpi delete_vm
method.
In the following scenario, the vcloud cpi fails to perform the delete vms, and would leave a leaking vapp:
1- a vm is requested for creation, but its IP is conflicting with an existing running vm in another vapp. The vapp and vm creation succeed, but the start fails. The vapp remains in the partially powered state.
<Error minorErrorCode="BAD_REQUEST" message="The following IP/MAC addresses have already been used by running virtual machines:
MAC addresses:
IP addresses: 192.168.26.215
Use the Fence vApp option to use same MAC/IP. Fencing allows identical virtual machines in different vApps to be powered on without conflict, by isolating the MAC and IP addresses of the virtual machines." majorErrorCode="400"/>
2- the vapp is asked for deletion by the CPI, but vcloud is refusing the delete request because the vm is in the partially started state.
D, [2015-07-15 15:08:10 #9434] [create_vm(9cbce686-08ac-498a-97de-79d99a4a97ce, urn:vcloud:catalogitem:740dbf1b-f5bb-41ee-8272-71bcc90ffc24, {"cpu"=>1, "disk"=>4096, "ram"=>1024}, ...)] DEBUG -- DirectorJobRunner: REST REQ DELETE https://.../api/vApp/vapp-e1d96da6-b607-4634-be46-9996067f1bfa {:Accept=>"application/*+xml;version=5.1", :content_type=>"*/*", :x_vcloud_authorization=>"...="} {"vcloud-token"=>"...", "Path"=>"%2F"}
D, [2015-07-15 15:08:11 #9434] [create_vm(9cbce686-08ac-498a-97de-79d99a4a97ce, urn:vcloud:catalogitem:740dbf1b-f5bb-41ee-8272-71bcc90ffc24, {"cpu"=>1, "disk"=>4096, "ram"=>1024}, ...)] DEBUG -- DirectorJobRunner: REST RES 400 {:date=>"Wed, 15 Jul 2015 15:08:10 GMT", :vary=>"Accept-Encoding", :content_type=>"application/vnd.vmware.vcloud.error+xml;version=5.1", :content_length=>"464"}
<?xml version="1.0" encoding="UTF-8"?>
<Error xmlns="http://www.vmware.com/vcloud/v1.5" minorErrorCode="BAD_REQUEST" message="The requested operation could not be executed on vApp "9cbce686-08ac-498a-97de-79d99a4a97ce". Stop the vApp and try again." majorErrorCode="400" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://[...]/api/v1.5/schema/master.xsd"></Error>
Currently in 0.7.10, the delete_vm
translates to a straight call to the vcloud api DELETE vapp . It should check the entity state to first power off the vms, prior to send the delete request
See vcloud REST API manual at http://pubs.vmware.com/vcd-55/topic/com.vmware.ICbase/PDF/vcd_55_api_guide.pdf p38
Undeploy, Power Off, and Delete the vApp
After you undeploy a vApp and power it off, you can use an HTTP DELETE request to delete the vApp object.
A deployed vApp has a link that you can use with a POST request to undeploy the vApp and take a power action such as powering it off or suspending it.
A powered-off vApp has a link that you can use with a DELETE request to remove the vApp.
When attempting to deploy a CF release in a VDC with "Number of resource intensive operations per user" and "Number of resource intensive operations per organization" both set to 10, the first 10 VMs are created, then deployment fails because the following error is received and not handled:
<Error
xmlns="http://www.vmware.com/vcloud/v1.5"
minorErrorCode="OPERATION_LIMITS_EXCEEDED"
message="The maximum number of simultaneous operations for user "***" on organization "***" has been reached."
majorErrorCode="400"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://***/api/v1.5/schema/master.xsd">
</Error>
I can't see a setting to tell the CPI what the limit is. Have I missed it, or is there another workaround?
While testing out the new vApp functionality, VM creation would sometimes fail due to timeouts (the subject of the previous issue raised). These incomplete VMs would remain after the failed deployment. That's OK, but I would expect these VMs to be cleaned up by then deleting the deployment.
Unfortunately, every time I ran "bosh delete deployment " it would complete successfully, but the vapp and VMs would still be there as before. BOSH Director thinks they're gone, but in fact they are not. I had to delete them manually in vCloud.
https://github.com/vchs/bosh_vcloud_cpi/blob/master/lib/cloud/vcloud/vcd_client.rb#L150-L157
If a vApp has had errors in the past (perhaps nothing to do with BOSH), it will have a failed task listed against it. That historical, irrelevant failed task will cause the current BOSH deployment to fail.
Proposed solution would be to take a timestamp at operation start, and ignore errors before that time?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.