Add description text to commands so the help is more helpful

source-control CD release steps

also should try moving execution of sleet.sh into azure container instances

Need to run functional test. A quick start for that would be to run all of the examples.

Show current version in command line title text. Probably based on the program assembly's AssemblyInformationalVersionAttribute.

https://github.com/Microsoft/Atlas/blob/806353a6b311eee6ac2bcadb4a3a975bcec0c851/src/Microsoft.Atlas.CommandLine/Program.cs#L46

Maybe also add a top-level --version or version command. Would be great if the version command showed the git commit along with the version number.

Simplest case is a path to folder of static file server, like: https://raw.githubusercontent.com/Microsoft/Atlas/master/examples/202-throwing-errors

Should recognize related folder https://github.com/Microsoft/Atlas/tree/master/examples/202-throwing-errors and change it to the related form automatically.

Maybe should also recognize if the path ends in /workflow.yaml and remove for convenience.

Need to create a workflow (like example 402) which controls the build and release definitions

A long object property will cause a leading ? to be written, which isn't valid json

The example which creates storage account, cdn, uploads files, etc has become very specific to the Atlas release pipeline.

It should probably be moved to a build subfolder instead. The example 402 can be simplified and show blob uploading and cdn creation without being an atlas release mechanism.

This would be a web app that uses something like Azure Container Instances to execute workflows

Should work like values but anything which evaluates to an array would cause the operation to run once for each array item.

Example usage:

operations:
- message: Listing subscriptions
  request: apis/azure/subscriptions-list.yaml
  output:
    subscriptions: (result.body.value)

- foreach:
    azure:
      subscription: (subscriptions[].subscriptionId)
  message: (['Listing resource groups in ', azure.subscription])
  request: apis/azure/resourcegroups-list.yaml
  output:
    results:
      (azure.subscription):
        resourceGroups: (result.body.value)

Task should also set the ATLAS_CONFIG_DIR environment variable in a way that ensures the credentials won't be used by other processes running on the machine.

Should also remove credentials which are added.

Not really sure what this one is. It might have something to do with when you choose to have Azure DevOps automatically manage the service principal credentials.

It becomes difficult very quickly to organize a large workflow into stages with child templates. Specific problems:

• The syntax for including an indented workflow fragment is complicated

    {{# indent depth=4 }}
    {{> child-template-name.yaml }}
    {{/ indent }}

• The same working memory is used for the whole workflow, making it possible to consume values or produce output unexpectedly
• Troubleshooting syntax errors becomes more difficult because the overall result becomes a single, large workflow file.
• Individual stages can't be run directly
• Workflow fragments often have re-usable logic. You end up with a situation where the same fragment file is copied into several locations, leaving you with the problem of keeping the

See Sub-workflows design docs for problem statement and solution proposal

With --dry-run switch operations can be skipped when the method is not GET. When that happens the output step for that operation shouldn't be executed.

Should work like the {{ json [expr] }} helper, where the first non-null expr is evaluation is serialized

Should also accept an option like {{ yaml [expr] indent=10 }} because yaml is sensitive to indentation.

The very first line will probably be an issue... Might not want to indent the very first line. Or might want to include a leading newline character to ensure every non-empty line has the same indentation.

Leading newline might be the way to go, actually, because that will also enable the helper to be used on a single line. That is to say, both of the following examples would have the same effect.

something:
  another: 
    {{ yaml foo indent=4 }}  

something:
  another: {{ yaml foo indent=4 }}  

https://github.com/Microsoft/Atlas/blob/c4cc4b0b1f84163b8b3816971705b07df640030f/examples/401-linux-agent-pool/scripts/setup.sh#L67

For example when invoking REST API on aks cluster, then the the apiserver endpoint certificate might need to be validated using the cluster-specific self-signed certificate authority.

There's a different way to fetch an access token when the pipeline task is given an Azure RM service endpoint that that is using msi-provided service principal.

Based on the autorest technique of using the readme.rm file as both documentation, description, and packaging metadata.

author: string
license: path
dependencies:
  swagger: { <mount-path> : [<swagger-reference>] }
  packages: { <mount-path>: <package-reference> }

On a linux vsts build agent the ansi color codes in the stdout are not being interpreted, and are showing up like 🔲[36m

Should figure out if they've come up with a better way with the latest .NET Core to generate color cross-platform than to write ansi to stdout.

One area that has been highlighted for getting started is to describe the various parts and files of a workflow and how they work together

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

When looping over data and adding values to a hash table it's very tricky to build that property-object relationship using to_object.

Better would be to be able to jmespath query property names in the same way as values.

  output:
    results:
      "(azure.subscription)":
        resourceGroups: (result.body.value[])

The query will need to result in a single string - if it is anything complex there's no way to know how to map what parts of the complex key should associate with what parts of the complex value.

Oh... I take that back. If the key results in an array of string and the value results in an equally sized array, it would be possible to use that information to turn it into a hash object.

  output:
    result:
      (result.body.value[].id): (result.body.value[])

The devops REST API calls should probably use the Visual Studio client id 872cd9fa-d31f-45e0-9eab-6e460a02d1f1 instead of the Atlas client id.

Possibly happened around the same time it was changed to have two tasks instead of one?

Maybe this is an optimization problem, or maybe it's a bad idea to read several dozen mb of binary files as a base64 encoded json property in order to upload it to blob storage.

We should consider adding a way to declare the body of an api as coming from a file instead of being declared as a binary json property. Something like a top level property like bodyFile or input.

Should also add a command-line switch named -i|--input in order to provide multiple base paths. Reading files which are not part of the template folder would be relative to one of those paths. It should also be impossible for a workflow to read any file that isn't a sub-path of an --input directory.

Currently a number of workflow make use of templating with a content.yaml file that has:
{{{ yaml content indent=0 }}}

Adding support for directly writing content to yaml would be a good improvement here

Probably means the response body shouldn't be the root data in a request operation's output:

Instead of (@) the body would be queried at (result.body), and (result.headers) and (result.status) would also be available.

Which should also have a nice side-effect of leaving the rest of working memory available as well.

The README.md text was written before the Azure DevOps branding was announced. There are a lot of places where it should be updated.

Some things might be a little tricky... Distinguishing between Azure DevOps the product, and DevOps the role... Also, if there are Azure DevOps API and Azure Active Directory API, what do you call the Azure API that's distinct? Azure Resources API? or Azure Resource Manager API?

In the build-ci.yaml file the script block has set -e which causes the step to fail when the first non-zero exit code is returned from a command.

This means the following line will correctly cause the build to fail if any unit tests fail:

docker run --name ${test_container_name} atlas-cli-test:0.1.$(Build.BuildId)

Unfortunately the following line must be run in order for the PublishTestResults task to be able to publish the trx files, which lets the build summary show test result details.

docker cp ${test_container_name}:/app/test/Microsoft.Atlas.CommandLine.Tests/TestResults/. $(Build.ArtifactStagingDirectory)/TestResults

It would be nice if the script run docker cp and docker rm commands even if the docker run failed dur to unit tests

The task needs to use the pem file in order to fetch a token for the workflow to run.

Token cache files should be isolated (different file names on disk) by various auth parameters.

• Different tenant (or authority) values should mean there is no overlap between credentials used.

• A different client app id should also mean the tokens have no overlap.

• The resource (or scope) values probably shouldn't have isolated token stores...

It was added recently, and it should be described along with build.sh

Should get ILogger welded into the console app for debug output. Consider things like --verbose should control is ILogger output should go to the console output and --debug should control if a logs folder is written to the _output folder.

Something you could use to run locally or in the cloud to run an Atlas workflow without installing atlas locally

The Azure task that can run in DevOps pipelines should be able to take several Azure resource manager and Azure DevOps service endpoints.

The task should call atlas account add several times so that the credentials are available during execution, and call atlas account clear afterwards to remove them.

It should also set the ATLAS_CONFIG_DIR environment variable in a way that ensures the credentials won't be used by other processes running on the machine.

(Edit: strikeout text is either done or moved out into different issues.)

Why is the tool package named dotnet-atlas when the tool is called atlas and doesn't actually execute as part of the dotnet CLI? As far as I can tell, atlas isn't limited to .NET projects and just happens to be written in .NET (awesome!). It's probably a good idea to use a name like atlas-cli if possible (and better to change this now than after a 1.0 ships ;))

I am facing a case where I am looping on a list of values (VSTS packages) and perform an API call on each (promote a certain package version in my case) . I would like to ignore the errors thrown by that API (when the package version combination doesn't exist) and proceed with the repeat loop instead.

The message "Determining latest version available" is displayed even if checkLatestVersion is false.

Maybe the if statement here https://github.com/Microsoft/Atlas/blob/ca858c166f43b2df3ec2f81f23b7a0ac80194aff/src/Tasks/AtlasInstallerV0/src/index.ts#L18 is wrong for boolean task properties?

Declaring request yaml can be inconvenient. Would be better to declare swagger file references which will auto-generated a request yaml file per operationId.

generate:
  swagger:
    {apis/path/prefix}: 
    - paths/to/swaggers.json
    - paths/to/folders[/readme.md] 

Swagger has a convention of title is client name and operationId is method name. So the generated files will have the effective path

api/path/prefix/title/operationId.yaml

Hello everybody,

I'd like to assert what's the current Atlas' status. The term, Atlas, is used in several contexts in Azure (for instance, MongoDB Atlas and atlas module for azure-maps-control package). It makes quite difficult to find consistent and updated documentation. Besides, considering that the newest commits are 4 years old, it makes me thing about Atlas' usage.

Does it worth using Atlas for deploying something in Azure? Is it "abandoned software"? Is it supposed to be still available, or is it deprecated?

By no means, these questions are sent to inquiry the project in itself. My intention is to verify the feasibility of using Atlas in the Microsoft Azure project (it was proposed as a requirement).

Best regards,
phanxen

it would be nice in an operation like this

- template: example-template.yaml
  write: example-output.yaml

if the write filename could be late-bound to variables (like message property can be)

- template: example-template.yaml
  write: ( ['example-', thing.name, '.yaml'] )

The render-time values can be incomplete when a model.yaml file is used to normalize defaults...

• The model.yaml should be merged onto incoming values as an overlay.

• The resulting merged values should be use for the initial template rendering as well as the initial runtime values

The path to the workflow location is special if it ends in .zip

Commands should operate on those paths exactly as if the file was an unzipped folder on disk.

Paths to zip on disk, and https urls with paths ending in .zip should both operate this way

Used to parse readme.md file for workflow yaml config