connector-iaas's Issues
Azure: Improve IP address attachement
Change the default behaviour to create a new IP address' resource only if desired (from new parameter).
Azure: script execution from remote file.
The Azure extensions mechanism allows to execute a script from a remote file (served through HTTP) in addition to a command.
The connector-iaas must include this feature to allow deeper VM customizations (eg. Java installation, creating custom firewall rules, etc.), such script files could then be shared via GitHub for example.
VMWare infrastructure: script execution fails on new instances
From 'executeScriptOnInstanceId' method, the VMWare call to 'startProgramInGuest' throws an exception.
This happens because new cloned VMs are not automatically powered on.
Error handling improvement
As pointed out by @lpellegr, we currently put 'ERROR' as a prefix every time we throw an exeption.
We should create our own RuntimeException for each CloudProvider, for instance in Azure we could use a custom AzureProviderException class and manage the prefix in this class at least to avoid some duplications.
Azure: Manage securityGroup types and specify it on deployment
It would be nice to specify what type of security group we want to create during instance creation.
Different types of security group are already defined into AzurePoviderUtils (Open/SSH/ProActive)
We thus need to specify the desired group from instance creation requests.
Azure: Manage disks size and disk attachement
- Specify the desired size of the OS disk to create (minimal value is the image size)
- Specify if we need to attach our new instance(s) to additional (and existing) managed disks
Instance(s) model improvement
The createInstance
method signature currently takes a single Instance with a field number
(number of instances to create) instead of multiple Instances.
This behaviour prevents single-instance customization which can be usefull for injecting custom script on each instance for example.
GET /infrastuctures returns sensitive fields like the credentials
Since there is no authentication to this service, the infrastructure endpoint shouldn't return the plain credentials, especially if the service is exposed publicly.
Provide a user authentication mechanism
We should have a way to secure the usage of the connector-iaas through a sessionid to list infrastructures, their instances and more particularly the execution of scripts on deployed instances.
Azure: Reusing existing resources
- Attach to an existing public IP address (if not already attached!)
- Attach to an existing security group
- Attach to an existing virtual network
AWS/JClouds: region 'us-east-2' not configured
When we try to access resources in the region us-east-2
(Ohio) we get the following error:
requested location us-east-2, which is not in the configured locations: {eu-west-1=Suppliers.ofInstance(https://ec2.eu-west-1.amazonaws.com), ap-northeast-1=Suppliers.ofInstance(https://ec2.ap-northeast-1.amazonaws.com), sa-east-1=Suppliers.ofInstance(https://ec2.sa-east-1.amazonaws.com), ap-southeast-1=Suppliers.ofInstance(https://ec2.ap-southeast-1.amazonaws.com), ap-southeast-2=Suppliers.ofInstance(https://ec2.ap-southeast-2.amazonaws.com), eu-central-1=Suppliers.ofInstance(https://ec2.eu-central-1.amazonaws.com), us-east-1=Suppliers.ofInstance(https://ec2.us-east-1.amazonaws.com), us-west-1=Suppliers.ofInstance(https://ec2.us-west-1.amazonaws.com), us-west-2=Suppliers.ofInstance(https://ec2.us-west-2.amazonaws.com)}
NPE in getAllInfrastructureInstances when retrieving bad VM config
In https://github.com/vinseon/connector-iaas/blob/master/src/main/java/org/ow2/proactive/connector/iaas/cloud/provider/vmware/VMWareProvider.java#L144, vm.getConfig() can be null in some cases when a VM is not correctly setup.
GET instances fails when busy creating or deleting instances
After sending a DELETE connector-iaas/infrastructures/<my_infrastructure_id>/instances?instanceTag=<my_instance_tag>
and while the request is being processed (which can take a lot of time), <my_infrastructure_id>
is not pollable anymore through GET or POST. Instead, I get a Internal Error 500.
AWS: delete EBS volumes along with VMs
In AWSEC2JCloudsProvider, when we create a new Instance we do not provide any information about the EBS volume that will be attached to the new VM. Therefore, by default, JClouds creates a gp2
volume (regular SSD) with the same size than the source image.
According to Amazon, when a new EBS volume is created, the flag deleteOnTermination
is set to true
but it seems that it is not the default behavior of JClouds.
The VM's template must therefore be cutomized with specific volume informations, see the method mapNewVolumeToDeviceName for instance.
Without this fix we will continue to pay for the remaining volumes after VMs deletion!
Azure: Non stable Azure SDK & thread safety
We currently use the last release (but still non stable) version '1.0 beta 5' of Microsoft Azure Java SDK.
Indeed, Microsoft announced that it is only a developer preview but which already supports major parts of the SDK.
Regarding thread-safety, it seems there is no garantee:
We do not make any thread-safety guarantees about our libraries. We also do not test them for thread-safety.
Methods that are currently thread-safe may be thread-unsafe in future versions.
The AzureProvider class has been however tested by us and the SDK classes we are using are actually thread safe.
However this need to be carefully double-checked after every SDK upgrades, as mentioned by Microsoft devs.
Azure: create VMs from native images.
Currently, only custom images can be deployed on Azure by specifying the image name/id. The Azure SDK allows to create VMs from native images like Ubuntu, Debian, or Windows. Such an option should be provided by the connector-iaas.
Azure: Improve script execution
Multiple init scripts:
We currently combine all scripts (from an InstanceScript object) to execute them at once. This is actually required for 'initScript' only (from createInstance method) as we can only give one extension per type during the VM creation.
Advanced script upload:
Support advanced scripts by specifying and uploading file-based scripts into Azure environment (can be then used by the Azure extension). The script can be then executed from a regular cmdLine injection.
Windows compatibility:
Check deployment and script execution on Windows (using the same extension than Linux). Check combined script as well using the '&&' separator.
Encryption:
Allow to send encrypted scripts on VMs using specific credentials (Azure extension feature)
Azure: allow PNP communication using resource manager
Unlike Amazon, Azure doesn't provide a complete metadata service nor automatic DNS setup for the VMs.
Therefore, a VM doesn't known its own public IP address and PNP communications can not be fully established between server and nodes.
Solutions could be to:
- Manually setup a DNS using Azure API and specify it to the VM by script injection
- Retrieve the VM public IP address using API and specify it during script injection
- Let the VM discover its own public IP address by using public service like
curl ipinfo.io/ip
(from script injection OR init script)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.