robe070 / cookbooks Goto Github PK
View Code? Open in Web Editor NEWThis project forked from lansalpc/cookbooks
Cookbooks for scripting LANSA Stacks
This project forked from lansalpc/cookbooks
Cookbooks for scripting LANSA Stacks
The PRD describes the following objective in section 6.1.4:
Following Azure Security Best Practices and utilizing the Azure Key Vault, design a secure mechanism to store and retrieve the Scalable License Certificates. To be designed by the architect before Phase 1 begins.
Here is a document that contains the design. And in particular the change required in the script. It will be augmented later with the Splash Screen design.
Refer here for details: https://www.dropbox.com/s/4vm4vep81a7u7r5/Cloud%20Deployment_%20Azure%20Scalable%20Images_Architect%20Prereqs%20-%20Google%20Docs.pdf?dl=0
Bake-Ide-Ami script should accept a [switch] Pipeline to handle the scenarios relevant to the pipeline build. This issue covers a scenario to suppress the Messagebox for the Pipelines.
Windows Server 2012 LANSA Scalable License image for V14 SP2 on Microsoft Azure Marketplace.
Deliver a Windows Server 2012 LANSA Scalable License image for V14 SP2 into the LANSA Azure account and provide Azure Marketplace Submission URLs to the LANSA Architect, This is an interim deliverable, until the image is successfully published in Azure Marketplace the deliverable is not complete.
ref PRD 18.9.2
Use Pester to write down the tests based on the PS scripts. The result of this task will be the Pester Test Script. This script when executed should generate a NUnit.xml file. Eventually, it will be used by the Azure Pipeline. It should include the TestImage tests and Test Image using Stack Test.
The purpose of this task is to test template post-deployment to stack. Thus, automating the task done in #34
Gather some of the performance related settings into a single option so that its a simple choice of, say, a small, medium or large stack.
Needs specifics to be designed.
Example settings are:
Application Gateway
Database
Instance counts
Combine with #17
Any edited scripts need to be checked to ensure that Write-Output
is replaced with Write-Host
and that every command line whose output is not captured to a variable is piped using |Out-Default | Write-Host
. Ref: Appendix: Generic Script Updating Requirement
The Image Release Pipeline builds a single image, use the table provided in the #4 to extend the build process to the six images.
The purpose of this issue is to provide, at the earliest oppourtunity, a submission to AMP that is valid, so that basic issues of publishing a Solution Template may be solved. The CreateUIDescription.json
file may be as simple as necessary to pass the tests.
Initial submissions to AMP can take a number of weeks, hence the need to get the process started ASAP.
Ref: #24
Requires the image to be published. Architect will notify when thats complete. This will be available by Monday 10th August.
Create a stack in US East and run the tests described in #7, but on the Production image, NOT Preview and NOT Custom Image
This should take about 1 hour with most of the time just waiting for the stack to be created.
Provide Azure Marketplace Package file for Solution Template to LANSA Architect.
This is an interim deliverable, until the Solution Template is successfully published in Azure Marketplace the deliverable is not complete.
One of the tasks is to automate the baking of the image. And there is currently a prompt which requires manual intervention. This exists because the script installs a Windows Update and that is not permitted using Remote Powershell.
After talking with Harish last night I wondered if the Az Module provided a cmdlet which would execute a powershell script in the VM. And there is. So you will replace the manual prompt by using this cmdlet:
https://docs.microsoft.com/en-us/azure/virtual-machines/windows/run-command.
This is optional for Phase 1 and required for Phase 2. I suggest you implement for phase 1 because it sounds simple to do and will speed up your work.
The purpose of this task is to allow for images in all stages of development, test and production to be used by the template.
This issue is required to be done at the earliest oppourtunity so that the template may be used to test the new images. The template will not be published yet, so it will need to be downloaded as a raw file from the GitHub repo. These changes should be made in lansa/azure-quickstart-templates
feature/azure-images
.
It must also incorporate the ability to use a Custom Image instead of a Marketplace image, again, so that a Custom Image may be tested by the pipeline. This will involve extending the template sample provided below to prompt for Marketplace Image (default) or Custom Image. And provide extra parameters to specify those required for a Custom Image.
Also change the Offer and SKU selection to get the SKUId and OfferId based on the user's selection of the OS, Lansa Version and Release State.
Sample Input 1
Sample Output 1
Sample Input 2
Sample Output 2
Refer the section 21.1 in the PRD.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"osName": {
"type": "string",
"allowedValues": [
"Windows Server 2019",
"Windows Server 2016",
"Windows Server 2012"
],
"defaultValue": "Windows Server 2019",
"metadata": {
"description": "Choose the Windows Server image version to use "
}
},
"lansaVersion": {
"type": "string",
"allowedValues": [
"V15 GA",
"V14 SP2"
],
"defaultValue": "V15 GA",
"metadata": {
"description": "Visual LANSA Version"
}
},
"imageReleaseState": {
"type": "string",
"allowedValues": [
"Production",
"Preview"
],
"defaultValue": "Production",
"metadata": {
"description": "Release state of the LANSA Scalable License image.The Preview release state is restricted to a very limited audience."
}
}
},
"variables": {
"marketplaceSettings": {
"Windows Server 2019": {
"offer": "lansa-scalable-license",
"version": {
"V15 GA": {
"sku": "win-2019-15-0"
},
"V14 SP2": {
"sku": "win-2019-14-2"
}
}
},
"Windows Server 2016": {
"offer": "lansa-scalable-license",
"version": {
"V15 GA": {
"sku": "win-2019-15-0"
},
"V14 SP2": {
"sku": "win-2019-14-2"
}
}
},
"Windows Server 2012": {
"offer": "lansa-scalable-license",
"version": {
"V15 GA": {
"sku": "win-2019-15-0"
},
"V14 SP2": {
"sku": "lansa-scalable-license-14-2"
}
}
}
},
"isPreview": "[equals(parameters('imageReleaseState'), 'Preview')]"
},
"resources": [],
"outputs": {
"offerId": {
"value": "[concat(variables('marketplaceSettings')[parameters('osName')].offer, if (variables('isPreview'), '-preview', ''))]",
"type": "string"
},
"skuId": {
"value": "[variables('marketplaceSettings')[parameters('osName')].version[parameters('lansaVersion')].sku]",
"type": "string"
}
}
}
Refer #4 for parameter mapping.
Alter the Default LANSA MSI Depending on LANSA Version
The default value used for the parameter msiURL
must change based on which LANSA Version is chosen.
LANSA Version | msiURL |
---|---|
V14 SP2 | https://lansalpcmsdn.blob.core.windows.net/app/test/AWAMAPP_v14.2.20715_en-us.msi |
V15 GA | https://lansalpcmsdn.blob.core.windows.net/app/test/AWAMAPP_v15.0.0_en-us.msi |
It would be ideal if this change to the default was in the template itself, and if thats not possible the behaviour may be implemented only in createUIDefinition.json
. And the V15 GA url would be the default in the template.
Both maintemplate.json
and createUiDefinition.json
need this behaviour. It is for end users as well as developers.
A technique that may work in the template is to use a logic function to select the default value. And that may work in the createUIDefinition.json
file too, otherwise hiding/showing copies of the same parameter may work, each with a different default.
Any edited scripts need to be checked to ensure that Write-Output
is replaced with Write-Host
and that every command line whose output is not captured to a variable is piped using |Out-Default | Write-Host
. Ref: Appendix: Generic Script Updating Requirement
Modify the Bake-Ide-Msi.ps1
script to create a resource group based on the VersionText, if it doesn't already exist.
This is important for the pipeline so that it may be cleaned up after building the image.
Bake-Ide-Msi.ps1
must not delete the resource group itself. During development its up to the developer to delete when appropriate. The pipeline will delete it on successful completion. Note that the image must be placed in the existing BakingDP resource group so that it is not deleted.
The KeyVault will stay in BakingDP
The image baking resource groups will be named BakingDP-<VersionText>
. E.g. BakingDP-w12r2d-14-2-10
Using the technique in the reference document to determine what is currently available everywhere, update the VM sizes in the template to these VM sizes which are available everywhere.
This task should be repeated every 6 months to confirm that the defaults are still valid. To that end, when this task is complete, create another Issue to perform the same task and note the date in the Issue. If possible set a reminder.
Design a pipeline refer the wiki page for more information, need to add subtasks.
Deliver a Windows Server 2012 LANSA Scalable License image for V15 GA into the LANSA Azure account and provide Azure Marketplace Submission URLs to the LANSA Architect, This is an interim deliverable, until the image is successfully published in Azure Marketplace the deliverable is not complete.
Design a pipeline refer the wiki page for more information, need to add subtasks.
Please find attached the log of the report after running the ARM TTK. I will link the steps to run the ARM TTK in Development environment in the Wiki. Below tests are failing that needs attention.
[-] apiVersions Should Be Recent (315 ms)
##[error] Microsoft.Sql/servers uses a preview version ( 2018-06-01-preview ) and there are more recent versions available.
Valid Api Versions:
2019-06-01-preview
##[error] Api versions must be the latest or under 2 years old (730 days) - API version 2018-06-01-preview of Microsoft.Sql/servers is 796 days old
Valid Api Versions:
2019-06-01-preview
##[error] Microsoft.Sql/servers/databases uses a preview version ( 2018-06-01-preview ) and there are more recent versions available.
Valid Api Versions:
2019-06-01-preview
##[error] Api versions must be the latest or under 2 years old (730 days) - API version 2018-06-01-preview of Microsoft.Sql/servers/databases is 796 days old
Valid Api Versions:
2019-06-01-preview
##[error] Microsoft.Sql/servers/firewallRules uses a preview version ( 2018-06-01-preview ) and there are more recent versions available.
Valid Api Versions:
2019-06-01-preview
##[error] Api versions must be the latest or under 2 years old (730 days) - API version 2018-06-01-preview of Microsoft.Sql/servers/firewallRules is 796 days old
Valid Api Versions:
2019-06-01-preview
Ref: Baked Image Testing
Add options to script for Application Gateway and when creating a new database to use settings appropriate for Development. For example, the Application Gateway would be small and 1 instance. The database would be General Purpose, Serverless, with 1.5 <= vCpus <= 12 with Gen5 hardware.
Simlarly for Test.
Modify template to output the image version and then check that the image in the VMSS matches the expected image: Offer, SKU and Version.
There is a useful tool that Azure provides to query the properties of a deployment: https://resources.azure.com/
Navigate down to your subscription, resource group and then to VMSS. You will find JSON similar to the extract at the end of this entry. You can see it clearly provides all the information we require to validate that the image is the correct one:
"imageReference": {
"publisher": "lansa",
"offer": "lansa-scalable-license",
"sku": "lansa-scalable-license-14-2",
"version": "14.2.10"
},
The Offer and SKU are already output because they are derived values..
To reference the version property in the outputs section use syntax similar to the following:
"imageVersion": {
"condition": "[not(variables('isCustomImage'))]",
"type": "string",
"value": "[reference(variables('dbvmssName')).storageProfile.imageReference.version]"
},
And for completeness, also output the Custom Image inputs.
Following is the VMSS output from https://resources.azure.com/
{
"value": [
{
"name": "dbrobgyo4_0",
"id": "/subscriptions/739c4e86-bd75-4910-8d6e-d7eb23ab94f3/resourceGroups/robg/providers/Microsoft.Compute/virtualMachineScaleSets/dbrobgyo4/virtualMachines/0",
"type": "Microsoft.Compute/virtualMachineScaleSets/virtualMachines",
"location": "australiaeast",
"plan": {
"name": "lansa-scalable-license-14-2",
"publisher": "lansa",
"product": "lansa-scalable-license"
},
"instanceId": "0",
"sku": {
"name": "Standard_B2ms",
"tier": "Standard"
},
"properties": {
"latestModelApplied": true,
"vmId": "3d666b90-b6c9-426b-ab54-bb07fb74ea03",
"hardwareProfile": {},
"storageProfile": {
"imageReference": {
"publisher": "lansa",
"offer": "lansa-scalable-license",
"sku": "lansa-scalable-license-14-2",
"version": "14.2.10"
}
[snip]
}
]
}
The script which copies the appropriate file to the target destination is ./scripts/install-lansa-scalable.ps1
This line of code:
copy-item "$Script:GitRepoPath\Marketplace\LANSA Scalable License\ScalableStartHere.htm" "$ENV:ProgramFiles\CloudStartHere.htm" | Out-Host
Needs to use Cloud and version major and version minor from registry to access from correct folder
If file does not exist in the derived path, then default to the file in the parent directory. E,g .\Azure\ScalableStartHere.htm
Ref: Testing Image and Template Before Submission to AMP
Part of Deliverable: Script to start up a stack and test that the application functions properly when deployed locally and through Azure Marketplace when in preview
Provide a summary of both the Image pipeline usage and the Template pipeline usage, in terms of what triggers what and the manual checks and interventions required.
Deliver Windows Server 2016 LANSA Scalable License images for V14SP2 and V15 GA into the LANSA Azure account and provide Azure Marketplace Submission URLs to the LANSA Architect; this is an interim deliverable, until the images are successfully published in Azure Marketplace the deliverable is not complete.
The generic build script should accept the below parameters.
VersionText
VersionMajor
VersionMinor
AmazonAMIName
GitBranch
Cloud
Win2012
Refer #4
This is the third part of producing the new Solution Template, final submission. The new UI interface blades.
The sequence of events around creating the Solution Template are summarised below
#24 describes the UI process at a high level, requiring Architect involvement in what parameters will be on what blades.
The Solution Template is fully regression tested in the build pipeline. (Previously it was going to be in preview which presumed that Microsoft will have checked the submission thoroughly by the. But they don't. They only do that when choose to go LIVE! Actually getting to preview seems to just require that the ARM TTK tests pass, which we already know is the case. Better to test it automatically in the build pipeline rather than have an extra manual step in Preview. Now, once its ready to submit for preview there are no further tests to be performed. From then on its just about Microsoft validation)
(Note: tables are edited in this Google Docs document. And permission needs to be provided to edit it)
Usage=test-temp
. These resource groups will be deleted at the COB IST. If they need to be retained overnight then the tag ShutdownPolicy=excluded
must be added manually.The tests are as follows
Test | SQLAZURE1 | SQLAZURE2 | SQLAZURE3 | SQLAZURE4 | SQLAZURE5 | SQLAZURE6 |
---|---|---|---|---|---|---|
AG Tier | Standard | Standard | Standard | WAF | WAF | WAF |
AG SKU | Standard_Small | Standard_Medium | Standard_Large | WAF_Medium | WAF_Large | WAF_Large |
DB New | New | New | New | New | New | New |
DB Type | SQLAZURE | SQLAZURE | SQLAZURE | SQLAZURE | SQLAZURE | SQLAZURE |
Edition | Basic | Standard | Standard | Standard | Premium | Premium |
RSO Name | Basic | S0 | S4 | S12 | P1 | P11 |
Image Offer | lansa-scalable-license | lansa-scalable-license | lansa-scalable-license | lansa-scalable-license | lansa-scalable-license | lansa-scalable-license |
Image Plan | lansa-scalable-license-14-2 | w19d-14-2 | w16d-14-2 | w19d-15-0 | w16d-15-0 | w12r2d-15-0 |
Result | Success | Success | Success | Success | Success | Success |
The tests for MYSQL and MSSQLS must programmatically create the database in the resource group before deploying the template.
Test | MYSQL1 | MSSQLS1 |
---|---|---|
AG Tier | Standard | Standard |
AG SKU | Standard_Medium | Standard_Medium |
DB New | Existing | Existing |
DB Type | MYSQL | MSSQLS |
Edition | Standard | Standard |
RSO Name | S2 | S2 |
Image Offer | lansa-scalable-license | lansa-scalable-license |
Image Plan | lansa-scalable-license-14-2 | lansa-scalable-license-14-2 |
Result | Success | Success |
Demo of Azure image building and testing automation pipeline to validate that images have been built correctly.
Ref: Testing Preview Template or Preview Image
lansa-scalable-license-preview
. If the image is in production then the Offer will be lansa-scalable-license
. If its a Custom Image the developer will need to ascertain themselves what the values should be. The SKU is the text version of the Plan Id. Presumably the Powershell cmdlet will use the Plan ID. For the current image the Plan ID does not match the image name. The Plan ID is lansa-scalable-license-14-2
. For all other images it is the Image Name minus the version number. Refer to the table in #4 for the mapping.Refer to section 21.1 ARM Template Parameter Mapping Technique in the PRD. This table replaces the table there and the technique details will need to change in accordance with this table.
The Image Name is restricted to 15 characters - its a limitation of Windows computer names. It also needs to be different from the previously published image hence the "-1" on the end. So I've shortened the Plan to allow for extensions like w191909-14-2-10 for the image name.
Windows Version | LANSA Version | Offer | Plan (aka SKU) | Image Name |
---|---|---|---|---|
Windows Server 2019 Datacenter | V14 SP2 | lansa-scalable-license | w19d-14-2 | w19d-14-2-1 |
Windows Server 2016 Datacenter | V14 SP2 | lansa-scalable-license | w16d-14-2 | w16d-14-2-1 |
Windows Server 2012 R2 Datacenter | V14 SP2 | lansa-scalable-license | lansa-scalable-license-14-2 | w12r2d-14-2-10 |
Windows Server 2019 Datacenter | V15 GA | lansa-scalable-license | w19d-15-0 | w19d-15-0-1 |
Windows Server 2016 Datacenter | V15 GA | lansa-scalable-license | w16d-15-0 | w16d-15-0-1 |
Windows Server 2012 R2 Datacenter | V15 GA | lansa-scalable-license | w12r2d-15-0 | w12r2d-15-0-1 |
Refer PRD section 14 Phase 2 Guidance
This is a new process for LANSA so direct guidance on how to develop a Solution Template is not possible. Nonetheless, essentially a Solution Template is an Azure Quickstart Template with added features.
So you will need to either already be an expert in Azure Quickstart Templates and Solution Templates or you will need to educate yourselves and raise your skill level to that of an expert. Liaising with the Architect and frequently pushing changes, at least weekly, to the git repo are essential to ensure that Phase 2 continues along the intended track.
There will be a daily meeting with the architect until it needs to change.
This describes the current structure of the Azure Quickstart Template and the process of taking it as it is and publishing it as a Solution Template. It is meant for information only to get an overview of what is currently there and how the template needs to be altered to be admissible to Marketplace. It is NOT the specification of the work required.
Marketplace Sample
It is mandatory to watch Building Solution Templates and Managed Applications for the Azure Marketplace. It is expected that the guidance provided in the video will be followed exactly unless exceptions have been authorized.
One of the main tasks in Phase 2 is to integrate the 'tests' folder that is in the Azure Quickstart Template repository into the robe070/cookbooks repository. The tests folder has been redesigned and is now called the ARM Template Test Toolkit. That link describes the automated testing that needs to be achieved in Phase 2. Both being able to run these tests on a developer's machine and integrated into Azure Pipelines.
Other reference documents are provided below. At least read the contents of every document to know what help is available to you.
This is an incomplete list of the differences between Azure Quickstart Templates and Solution Templates
azuredeploy.json
=> mainTemplate.json
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for the resources."
}
}
Ref: https://github.com/Azure/azure-quickstart-templates/blob/master/Deploy-AzureResourceGroup.ps1
Implement at least the Dev
and ValidateOnly
switches. Dev
switch to be able to simply switch to using the test.parameters.json
file.
UI Experience
CreateUIDefiniton.json
file to specify blades. Architect will assist with appropriate naming of the blades and grouping of the parameters.To package up the Solution template for submission to AMP, zip up the createUIDefinition.json file and mainTemplate.json
file.
If the publish fails due to PR feedback, clicking on the link may result in a 401. Request Support to give you access.
Last resort email addresses:
[email protected]
[email protected]
Ref Issue #4
Windows Server 2012 R2 Datacenter | V14 SP2 | lansa-scalable-license | lansa-scalable-license-14-2 | w12r2d-14-2-10 |
---|
Note: There are 3 public ip addresses. Only the Application Gateway public ip address needs to be altered. This is because it is the address of the application that is being deployed. Whereas the other 2 are for troubleshooting the VMs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.