qualisystems / azure-shell Goto Github PK

-- Limit for Network Security Groups (NSG) soft: 100, hard: 400 - we use 2 NSGs for QX & ES.

-- limit for Private IP Addresses per virtual network is 4096 ==> Concurrent apps: 4096 in CloudShell

-- Network Interfaces (NIC) soft: 300, hard: 10000

-- Concurrent Public IP addresses (dynamic) - 60 (soft) - need to contact support to increase, no hard limit.
Concurrent Reserved public IP addresses (static) 20 (soft) - need to contact support to increase, no hard limit

-- NSG rules per NSG - soft: 200, hard: 500 (if we have 2 rules per app on avg than we can have a max of 225 apps in a sandbox)

-- Cores quota - default is 30 per region, need to contact support to increase, max is 10K.

-- Number of storage accounts per subscription - soft: 200, hard: 250 - max number of sandboxes is 249 (one storage account is always used by management stuff). if we will add the support for premium storage accounts than the max number of sandboxes will be 124 (like in aws)

*** Concurrent sandboxes: 249 ***

-- Extension script must be uploaded to a public location by the app designer, where in the azure portal the user only needs to browse for the file and azure will save it somewhere accessible by the VM.

Maybe user will copy the name of the image from the portal and we will try to match it to the best image we can find

• Need to find the right subnet that was created for the sandbox during setup "prepare connectivity" phase
  and connect the deployed vm to the subnet.
• If an app is added to an active reservation (that doesn't have a subnet), when trying to deploy return an error message (same as AWS)
• If a subnet exists - deploy and connect to that subnet.

Checking the subnet is done by connecting to the Azure API and retrieving the info according to the reservation id tag.

-In the deploy_azure_vm when we are creating a new client (using the Azure client factory) we are creating it for each and every operation .this needs to be done only for the 1st call for a method.

-Another thing that we need to check is that when changing the credentials for the client
the creation of client needs to take it into account.
(we should check that the session created with the given credentials).

Need to update the NSG rules in case the public IP was changes and there are open ports for the relevant VM

vCenter
AWS
Azure
OpenStack

The code should be backwards compatible

We need to let the user define custom tags. Up to 9 custom tags. Because Azure limit is 15 and we already use 6 tags for every object.

The custom tags should be applied to the VM and all related objects that are created during the app deploy

Add an attribute "Custom Tags" that will support the following syntax (comma separated list):
[TagName]=[TagValue], [TagName]=[TagValue]
Example:
Tag1=Val1, Tag2=Val2

When deleting a VM that has Inbound Ports we need to remove the relevant rules from the NSG.

STR
Go to manage -> apps -> Add
Choose Azure app

BIR
You can't choose the relevant cloud provider in the drop down list

1. Move all operation creation to the ctr
2. move all parser calls under the error handling context manager
3. Make sure all commands have an error handling context manager

In the prepare stage we need to generate OpenSSH keypair (public & private keys) and save it in the relevant azure storage.

1. Create a new domain service to work with OpenSSH keypairs. The service should return an object with private and public keys.
2. Extend the storage service to support saving and reading the keypairs.

Example from StackOverflow how to generate a keypair:
http://stackoverflow.com/questions/2466401/how-to-generate-ssh-key-pairs-with-python

Azure guide on generating keypairs for general knowledge:
https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-mac-create-ssh-keys/

• How to test that the key is generated correctly?
  Try to use the generated key when creating a linux VM in the azure portal. If Azure accepts the key than its ok.

an example script that gets all the inputs it needs (Azure region, VirtualnNetwork, Subnet, VM image, credentials...) and deploys a VM in Azure

the goal is to understand the attributes and inputs we need for Azure API, how the credentials flow is working

nothing should be modeled in cloudshell at this stage (not the "deployed app" and not the "Azure" cloud provider)

Refresh IP command should be available on deployed apps. (Connected command from the Azure resource)

Input: None

The command should connect to the Azure API and retrieve the private and public IPs, and then update the resource details in CloudShell.

Private IP - the resource address
Public IP - attribute on the deployed app resource

• need to configure the NSG rules in case the public IP was updated if there are open ports - moved this part to a different story #63

a hidden Delete command (interface should be similar to AWS and vCenter) that deletes the VM only from Azure

the flows that should be tested here are:

1. removal of VM from reservation should delete it from Azure and CloudShell
2. teardown should delete the VM from Azure and CloudShell
3. deletion from CloudShell inventory should delete the VM from Azure and CloudShell
4. test the scenario in which the VM is already deleted from Azure, shouldn't fail

notes:

• we shouldn't fail the command if the VM is already deleted
• sync, the command should be completed only once the VM is deleted
• no bulk support
• need to check how this story affects/relates to QualiSystems/AWS-Shell#123 , talk with Shay
• this command should support a case when no command runs without reservation id

-When storage with same name is created under 2 different resource groups we get an exception.

-We should create a storage with a Guid ,maybe with the name of the reservation.

In Azure we should use the resource group to delete all objects at once

As part of Azure installation guide we need to document how to configure Azure API with you AD user - see details here:
http://confluence.quali.com/display/QP/Azure+Configuration

• ability to create a Azure cloud provider in the Inventory (no need for shell template yet)
• new deployment option "Azure VM" with the required attributes

Flow that should be supported:

• > 1. create "Azure" cloud provider
• > 2. fill in "Azure" cloud provider attributes
• > 3. create new app and select "Azure VM" deployment option
• > 4. point to "Azure" cloud provider resource via attribute
• > 5. fill in deployment attributes - VM image, additional required parameters
• > 6. create an environment, add the app from the catalog and reserve
• > 7. setup should deploy the app
• > 8. once the app is deployed it should be already powered on and have its private ip as the address

Notes:

• > (a) the creation of the virtual network, subnet and SG is all in advance so we should have temporary attributes on the cloud provider resource with their information. --> prepare connectivity isn't implemented yet
• > (b) custom ports to open not supported in this stage
• > (c) public ip not supported in this stage
• > (d) all other commands (power on/off, delete etc.) should be mocked so Setup won't fail
• > (e) although the VM is powered on at the end of this flow the live status isn't updated yet, this will happen in #5
• > this story should include tagging, same tags as in AWS

data model (not finalized!) can be found here:
http://confluence.quali.com/pages/viewpage.action?pageId=18514287

We should create the Private IP as a dynamic IP add after it's created(after the VM is created) and the IP allocated we should change the IP to Static with the IP we got.

we need the ability to RDP/Telnet/SSH a Linux/Windows VM via QualiX, need to understand how the credentials mechanism works in Azure and figure out if there is a gap in QualiX

App configuration may have in the information about the OS creadentials for the App - as attributes on the Deployed App Model (available from the App advanced section).

For windows VMs:

• If the App request do not define the user & password -> VM credentials: u:adminuser p: [auto generated]
• If the App request defines the user (without password)-> VM credentials: u:[requested user], p:[auto generated]
• If the App request defines the user & the password -> VM credentials: u: [requested user], p:[requested password]
• If the App request defines the password (without the user)-> VM credentials: u:adminuser, p:[requested]

this is just a test

• * Manual configuration of the the management networking elements
• * Implement the prepare connectivity for Azure:
• * Create resource group for the sandbox
• * Create a storage resource for the sandbox
• * Attach the default tags to all sandbox resources
• * Create a key pair for the sandbox and save it in the storage resource
• * Create the subnet for the sandbox and the NSG object (configure the NSG to accept traffic from the management network), Deny traffic from other subnets in the sandbox vnet.
• *Add only a subnet to the management VNET(the VNET should be already configured) ,add attribute on Cloud Provider management_group_name. *
  In the management_group_name. we will find the Management VNET and its Tags ,Cidr
  It should contain the Tag: Role = "Sandbox" and "Management"

http://prntscr.com/cw9cko

Architecture:
http://confluence.quali.com/pages/editpage.action?pageId=26346704

ability to power on and power off an Azure VM

notes:

• need to update live status
• command should be sync, command should finish only once the vm is powered on/off

1. Add attribute to the deployment service "Inbound Ports" (it already exists in the AWS Shell)
2. Implement the same functionality like in the AWS Shell. The logic to parse already exists in the AWS shell.
   Need to determine the priority for new rule on the fly based on the next available priority. This section in the code should be atomic in order to guarentee a unique priorety per each rule when deploying.

• Source: "Any"
• Destiantion: Private IP of the deployed app, no need to use "/32" CIDR notation as azure accepts a single IP address
• Protocol: UDP/TCP from attribute value
• Port: from attribute value
• Priority: Minumun priorety 1000. Increase between rules in priority will be 5. Example: 1000, 1005, 1010...

Syntax for specifying ports is a semicolon separated list:
protocol[tcp(default)/udp]:port[single/range]

example: 80,443:tcp,200-220:udp

1. If no public IP than the NSG rule cannot be created. Need to add a valiadation before creating the VM and throw an exception if the "Inbound Ports" attribute is not empty and "Add Public IP" is false.

Note: "Outbound Ports" attribute is redundant

Have an environment with app
Reserve the environment

BIR
The app is deployed and gets his public IP as it address

The address of the resource should be the private IP and the public IP should be saved in an attribute "Public IP"

App configuration may have in the information about the OS creadentials for the App - as attributes on the Deployed App Model (available from the App advanced section).

For Linux VMs:

• If the App request does not define the user & password -> VM credentials: u:adminuser, p: not needed (connectivity by key pair)
• If the App request defines the user (without password)-> VM credentials: u:[requested user], p: not needed (connectivity by key pair)
• If the App request defines the user & the password -> VM credentials: u: [requested user], p:[requested password]
• If the App request defines the password (without the user)-> VM credentials: u:adminuser, p:[requested]

the key pair is generated in the 'prepare connectivity' phase and saved when creating a storage account for the sandbox.
When deploying the vm, the key pair that is passed to the Azure API should be taken from the sandbox storage resource.

Before we implement story #66 we need to test the copy speed of an image between two storage accounts.

Create a simple script that will copy a vhd blob with 30GB+ multiple times to test the performance.

https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-upload-vhd/

Related to #66

Waiting to get icons from Yoel

ability to define inbound/outbound port rules that will be applied to the Azure VM

in case an app has those port rules attribute populated a dedicated NSG should be created for this VM and associated with it as part of its deployment

note that in Azure you can define Allow and Deny rules. the user will define only Allow rules and we should make sure everything else is denied. this might be solved by the NSG of the Sandbox Subnet, need to check how NSG work in Azure.

the attributes that should be added to the deployment service (as user input = true) are:
a) Inbound Ports
b) Outbound Ports

Syntax for specifying ports is a semicolon separated list:
protocol[tcp(default)/udp]:port[single/range]

example: 80,443:tcp,200-220:udp

consider moving the parsing logic to a shared location, so we won't copy the AWS code

Check if its possible to create the API credentials using python/CLI. If not than we need an installation guide on how to do that.

http://confluence.quali.com/display/QP/Azure+API+Configuration

• CI
• branches (protected)
• code review
• basic project structure
• code coverage

1. Need to create a resource template JSON that will bootstrap our management network as described on the architecture page.
   http://confluence.quali.com/display/QP/Azure+Architecture+Design

Inputs:

• - Management Resource Group name (default value "Quali Management RG")
• - Mgmt VNET CIDR (default: 192.168.120.0/24)
• - CloudShell Server IP (for ES config)
• - CloudShell Server Username (for ES config)
• - CloudShell Server Password (for ES config)
• - On Premise CIDR (for VPN GW config)
• - VPN address (for VPN GW config)
• - VPN Shared Key (for VPN GW config. If empty need to generate a random key if possible)
• - QualiAgent Admin Username (default value: adminuser)
• - QualiAgent SSH public key
• - CloudShell Version (its a dropdown of supported versions)

Outputs:

• - Application Id (not implemented)
• - Application Secret (not Implemented)
• - VPN Gateway Public IP (only if the VPN will be selected)
• - QualiX Public IP

https://blogs.msdn.microsoft.com/girishp/2015/06/16/azure-arm-templatestips-on-using-outputs/

General notes:

• Sandbox VNET CIDR is 10.0.0.0/8
• QualiAgent size "Standard_A3" (need to review this with Assaf)
• Also create an application with the Contributor role for the use of our Shell
• Installation of ES & QualiX (should be provided by Alex Abugov)
• Add a Deploy to Azure button to our GitHub:
  • Explanation: https://azure.microsoft.com/en-us/blog/deploy-to-azure-button-for-azure-websites-2/
  • Example: https://github.com/Azure/azure-quickstart-templates/tree/master/101-mobile-app-create

Resources:

• Azure Resource Manager (ARM) Templates Visualizer - http://armviz.io
• ARM Templates examples: https://github.com/Azure/azure-quickstart-templates

Open questions:

1. How to use user data when deploying a VM
2. How to pass parameters from the template to the deployed VM (e.g : the address of the server).

Handle the creation of the IP (static \Dynamic)
if add public IP is set to true we need create an IP.

Deployment option attributes:
"Public IP Type" - string ("Static" or "Dynamic")
"Add Public IP" - bool, default value is False

• New deployment option for custom image
• Customer prepares the vm and uploads all custom images to a single storage account under the management resource group
• When deploying start copy the image to the reservation storage account. If the save image is used by more than one app than we need to copy the image only once.
• Instead of the 3 image identifier attributes that we have in the regular deployment service need to add an attribute "Image URN".

Article about Upload and create a Linux VM from custom disk image:
https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-upload-vhd/

Billing:

• Every VM will have tags
• Clicking ‘View My Bill’ from the Azure interface leads to the azure subscription bill
• Clicking the right subscription and then ‘Download usage details’
• From that page you can get a csv with two tables – 1. Billing Summary 2. Instance summary which contains the tags

http://stackoverflow.com/questions/38400740/how-to-delete-a-disk-with-azure-sdk-for-python

Redundant attributes that should be removed on Deployment Service:

• Outbound Ports

Redundant attributes that should be removed on Cloud Provider resource:

• Azure Mgmt Network ID
• Keypairs Location
• Storage Type
• AZURE MGMT VNET

Cloud Provider resource attribute type changes:

• Azure Secret - change attribute type to Password

Final attribute names and descriptions:

http://confluence.quali.com/pages/viewpage.action?pageId=18514287 (in progess)

We need to understand what is the expiration timeout for the Azure clients when idle.
And how to work with refresh token url.

• The Apps infrastructure now supports a special field for Cloud Provider.
• The existing shells (vCenter, AWS) include the cloud provider name / vcenter name as an attribute on all the deployment types.
• We need to align with the new infrastructure by refactoring all deployment types:

  Remove the Cloud Provider / vCenter name attribute from the shell package
  Update the drivers and scripts to take the value from the app context instead of the attribute.