Giter Club home page Giter Club logo

mongodbatlas-cloudformation-resources's Introduction

MongoDB Atlas AWS CloudFormation Resources & AWS Partner Solution Deployments

Code Health Contract Testing

Use AWS CloudFormation to manage MongoDB Atlas.

Partner Solutions (formally AWS Quick Starts) are automated reference deployments built by Amazon Web Services (AWS) solutions architects and AWS Partners. Partner Solutions help you deploy popular technologies to AWS according to AWS best practices. The quickest way to get started is to launch the official MongoDB Atlas on AWS Partner Solution Deployment directly from the AWS Management Console.

Getting Started

MongoDB Atlas CloudFormation

See the cfn examples to setup prerequisites and get started with your first cluster using our sample CloudFormation Stack templates.

MongoDB Atlas AWS CDK

See the cdk examples to setup prerequisites and get started with your first cluster using our AWS CDK sample code.

Limitations

  1. Resource import is not supported for third-party resources.

  2. Any third-party resource must support all CRUD operations. MongoDB Atlas for CloudFormation does not support the following MongoDB Atlas resources because they do not support all CRUD operations:

  • cloud-backup-snapshot-export-job
  • cloud-provider-access
  • federated-settings-identity-provider
  • federated-settings-org-configs

Support, Bugs, Feature Requests

Feature requests can be submitted at feedback.mongodb.com - just select "Atlas CloudFormation Resources" as the category or vote for an already suggested feature.

Support for the MongoDB Atlas Resource Provider for CloudFormation is provided under MongoDB Atlas support plans, starting with Developer. Please submit support questions within the Atlas UI. In addition, support questions submitted under the Issues section of this repo are also being monitored. Bugs should be filed under the Issues section of this repo.

MongoDB Atlas API Keys Credential Management

Atlas API keys Configuration are required for both CloudFormation and CDK resources, and this Atlas API key pair are provided as input by the use of a Profile

AWS CloudFormation limits Third Parties from using non-AWS API Keys as either hardcoded secrets in CloudFormation templates or via CDK, hence we now require all the users store MongoDB Atlas API Keys via AWS Secrets Manager.

NOTE: the process for configuring the PROFILE is the same and is required both for CloudFormation and CDK

1. Configure your MongoDB Atlas API Keys

You'll need to generate an API key pair (public and private keys) for your Atlas organization and configure them to grant CloudFormation access to your Atlas project. Refer to the Atlas documentation for detailed instructions.

2. Configure your Profile

To use Atlas CloudFormation resources, you must configure a "profile" with your API keys using AWS Secrets Manager.

The secret should follow this format:

SecretName: cfn/atlas/profile/{ProfileName}
SecretValue: {"PublicKey": "YourPublicKey", "PrivateKey": "YourPrivateKey"}

To create a new secret for a default profile, use the PROFILE SECRET TEMPLATE file provided in this repository.

Here are some examples of how to use this template:

Example 1

  ProfileName: default
  SecretName: cfn/atlas/profile/default
  SecretValue = {"PublicKey": "YourPublicKey", "PrivateKey": "YourPrivateKey"}

Example 2

  ProfileName: testProfile
  SecretName: cfn/atlas/profile/testProfile
  SecretValue = {"PublicKey": "YourPublicKey", "PrivateKey": "YourPrivateKey"}

Note: If you want to use a AWS KMS key to handle encyption of your secret you must define the appropriate resource policies so that AWS Secrets Manager has access to the key. The following AWS documentation describes a key policy that allows access through AWS Secrets Manager for all principals in the account that are authorized to use AWS Secrets Manager.

3. Provide the profile to your CloudFormation template

All Atlas CloudFormation resources include a "Profile" property that specifies which profile to use. You'll need to provide the profile you created in the previous step to the CloudFormation template.

Note that if you don't provide a profile, the resource will use a default profile (will try to get a secret named cfn/atlas/profile/default). We recommend always specifying the profile to avoid any unexpected behavior.

Once you've provided the profile, you can deploy the CloudFormation stack using the AWS Console or the AWS CLI. Refer to the AWS documentation for instructions on how to deploy CloudFormation stacks.

IMPORTANT: when specifying the profile in your CloudFormation template, you must specify the Profile Name, NOT the Secret Name

Correct usage:

  "Profile" : "ProfileName"

Incorrect usage:

  "Profile" : "cfn/atlas/profile/ProfileName"

Logging

Logging for AWS CloudFormation Public extensions is currently disabled. AWS is evaluating if logging is useful for consumers of third party extensions, if this is something you need or would like to request please open a ticket directly with AWS Support.

Contributing

See our CONTRIBUTING.md guide.

Issues

Autoclose stale issues and PRs

  • After 5 days of no activity (no comments or commits on an issue/PR) we automatically tag it as "stale" and add a message: This issue/PR has gone 5 days without any activity and meets the project's definition of "stale". This will be auto-closed if there is no new activity over the next 5 days. If the issue is still relevant and active, you can simply comment with a "bump" to keep it open, or add the label "not_stale". Thanks for keeping our repository healthy!
  • After 5 more days of no activity we automatically close the issue/PR.

One-click reproducible issues principle

Our support will prioritise issues that contain all the required information that follows the following principles:

  • We should be able to make no changes to your provided script and successfully run a local execution reproducing the issue.
    • This means that you should kindly provide all the required instructions. This includes but not limited to:
      • CFN resource template used to reproduce the issue
      • CFN resource version and AWS region used to reproduce the issue
    • Issues that cannot be properly reproduced will be de-prioritised in favour of the ones that succeed.
  • Before opening an issue, you have to try to specifically isolate it to the CFN MongoDB Atlas resource by removing as many dependencies as possible. If the issue only happens with other dependencies, then:
    • If other CFN resources are required, please make sure you also include those. Same "one-click reproducible issue" principle applies.
    • If external components are required to replicate it, please make sure you also provides instructions on those parts.

Troubleshooting

The following are common issues encountered when using AWS CloudFormation/CDK with MongoDB Atlas Resources:

  1. Activate the 3rd-party extension for each resource (i.e. MONGODB::ATLAS::[RESOURCE-NAME]) in each AWS region and from each AWS account that you wish to deploy.
  2. Ensure you have a sufficiently strong AWS IAM Activation Role attached to each 3rd-party extension.
    • For sample IAM Role see here
  3. Ensure your activated 3rd-party public extension matches name exactly to MONGODB::ATLAS::[RESOURCE-NAME] .
    • You might need to delete private extension if this namespace is already occupied.
  4. Ensure your MongoDB Atlas Programmatic API Keys (PAKs) being used with CloudFormation have sufficiently strong permissions (Organization Project Creator or Organization Owner)
  5. Ensure your MongoDB Atlas PAKs have correct IP Address / CIDR range access.
    • For testing purposes with caution you can open keys to all access by adding “0.0.0.0/1” and “128.0.0.0/1” (do not use for production workloads).
  6. How to determine which IP address AWS CloudFormation uses to deploy MongoDB Atlas resouces with my Atlas Programmatic API Keys (PAK)?
    • When you deploy MongoDB Atlas using CloudFormation with your Atlas PAK, CloudFormation will default to use the IP address of the machine that you are making the API call from.
    • The machine making the API call to the 3rd-party MongoDB Atlas API would be various AWS servers hosting Lambda functions and won't be static.
    • Review the AWS IP address ranges and contact AWS Support directly who can help confirm the CIDR range to be used in your Atlas PAK IP Whitelist.

Error: The CFN stack remains in the CREATE_IN_PROGRESS state before failing after an hour or so

The problem is caused by incorrect trust relationships linked to the role that you used to activate CFN resources or run the CFN stack. To resolve the issue, ensure that your IAM role's trust relationships include resources.cloudformation.amazonaws.com, cloudformation.amazonaws.com, lambda.amazonaws.com. The following YAML code shows an example of the correct trust relationships:

 AssumeRolePolicyDocument:
    Version: '2012-10-17'
    Statement:
    - Effect: Allow
      Principal:
       Service:
       - lambda.amazonaws.com
       - resources.cloudformation.amazonaws.com
       - cloudformation.amazonaws.com
      Action: sts:AssumeRole

Use the execution-role.yaml file to generate an IAM role that you can use to activate the CFN resources and run your CFN stacks.

Alternatively, you can set the trust relationships of your role via AWS Console: in the IAM dashboard, select your role and click Trust Relationships: Screenshot 2023-03-31 at 17 32 55

Error: 404 (request "INVALID_GROUP_ID") An invalid group ID |default was specified

The problem is caused by using the project resource identifier (ID + Profile Name) as the input parameter ProjectID of another CFN resource. The correct approach is to use GetAttr function to get the ProjectId from the project resource and use that value as input parameter to the next CFN resource. Example of correct use of GetAtt (see project-cluster.json):

   "AtlasCluster": {
      "Type": "MongoDB::atlas::Cluster",
      "Properties": {
        "ProjectId": {
          "Fn::GetAtt": [
            "Project",
            "Id"
          ]
        },
        "Name": {
          "Ref": "ClusterName"
        },

Error: The CFN stack failed to delete MongoDB::Atlas::NetworkContainer with the error 409 (request "CONTAINERS_IN_USE"). You cannot modify in-use containers; the container still contains resources."

The problem is that Atlas resources are using your network container. If your Atlas project has a cluster or a network peering resource, you can't delete the network container. To make sure your CFN stack deletes clusters and network peering resources before attempting to delete the network container, you should add DependsOn to your cluster resource.

Resources:
  Cluster:
    Type: 'MongoDB::Atlas::Cluster'
    DependsOn: NetworkContainer
    Properties:
      Profile: "default"
      Name: MyCluster
      .........
  NetworkContainer:
    Type: 'MongoDB::Atlas::NetworkContainer'
    Properties:
      Profile: "default"
      AtlasCidrBlock: 192.168.0.0/21
      RegionName: EU_WEST_1
      ProjectId: 'YOUR-PROJECT-ID'
  NetworkPeering:
    Type: 'MongoDB::Atlas::NetworkPeering'
    DependsOn: NetworkContainer
    Properties:
      Profile: "default"
      ProjectId: 'YOUR-PROJECT-ID'
      ContainerId: !GetAtt "NetworkContainer.Id"
      AccepterRegionName: "eu-west-1"
      AwsAccountId: "YOUR-AWS-ACCOUNT-ID"
      RouteTableCIDRBlock: "10.0.0.0/16"
      VpcId: "YOUR-VPC-ID"

mongodbatlas-cloudformation-resources's People

Contributors

adelmar92 avatar agustinbettati avatar andreaangiolillo avatar brianterry avatar codergo93 avatar colm-quinn avatar dependabot[bot] avatar elizhl avatar espenalbert avatar govardhanpagidi avatar jasonmimick avatar lantoli avatar maastha avatar marcosuma avatar marinsalinas avatar martinstibbe avatar oarbusi avatar parthasarathy-varadhan avatar parthasarathyvaradhan avatar pierwill avatar poornatejakonathampi avatar pvkrd avatar rdharmedran avatar shum avatar supermohit avatar themantissa avatar thetonymaster avatar vsnyc avatar zach-carr avatar zuhairahmed avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mongodbatlas-cloudformation-resources's Issues

MongoDB::Atlas::ServerlessInstance unable to output connection strings

The MongoDB::Atlas::ServerlessInstance schema is currently unable to output the connection strings.

Expected Behavior

Required to be able to export the srv connection strings for consumption in another template.

As per documentation i have the following setup:

AtlasServerless:
    Type: MongoDB::Atlas::ServerlessInstance
    DependsOn: AtlasProject
    Properties:
      ApiKeys:
        PublicKey:  !Ref "PublicKey"
        PrivateKey: !Ref "PrivateKey"
      ProviderSettings:
        RegionName: !Ref ClusterRegionUppercase
        ProviderName: "SERVERLESS"
      ProjectID: !Ref "AtlasProject"
      Name: !Sub ${EnvironmentName}-${BeName}-Serverless

Actual Behavior

The serverless instance creates fine, however when cloudformation attempts to build the outputs it fails

I have tried the following outputs with their associated errors listed below:

  1. As listed in Documentation:
  MongoDBAtlasConnectionStringsSrv:
    Condition: IsNotProduction
    Description: "Hostname for mongodb+srv:// connection string"
    Value: !GetAtt "AtlasServerless.ConnectionStrings"
    Export:
      Name: !Join [ ":", [ !Ref "AWS::StackName","ConnectionStringsSrv" ] ]

Error:
Unable to retrieve ConnectionStrings attribute for MongoDB::Atlas::ServerlessInstance, with error message The next fields are required ProjectID ApiKeys.PrivateKey ApiKeys.PublicKey

  1. As listed in Documentation:
  MongoDBAtlasConnectionStringsSrv:
    Condition: IsNotProduction
    Description: "Hostname for mongodb+srv:// connection string"
    Value: !GetAtt "AtlasServerless.SrvConnectionString"
    Export:
      Name: !Join [ ":", [ !Ref "AWS::StackName","ConnectionStringsSrv" ] ]

Error:
Requested attribute SrvConnectionString does not exist in schema for MongoDB::Atlas::ServerlessInstance

  1. Assumed meaning of documentation:
  MongoDBAtlasConnectionStringsSrv:
    Condition: IsNotProduction
    Description: "Hostname for mongodb+srv:// connection string"
    Value: !GetAtt "AtlasServerless.ConnectionStrings.SrvConnectionString"
    Export:
      Name: !Join [ ":", [ !Ref "AWS::StackName","ConnectionStringsSrv" ] ]

Error:
Requested attribute ConnectionStrings.SrvConnectionString does not exist in schema for MongoDB::Atlas::ServerlessInstance

  1. Derived from Atlas Cluster documentation and underling API
  MongoDBAtlasConnectionStringsSrv:
    Condition: IsNotProduction
    Description: "Hostname for mongodb+srv:// connection string"
    Value: !GetAtt "AtlasServerless.ConnectionStrings.StandardSrv"
    Export:
      Name: !Join [ ":", [ !Ref "AWS::StackName","ConnectionStringsSrv" ] ]

Error:
Requested attribute ConnectionStrings.StandardSrv must be a readonly property in schema for MongoDB::Atlas::ServerlessInstance

Cluster attribute not working: SrvAddress

I've modified the cluster.json example to add an output for SrvAddress:

    "Outputs": {
        "Id": {
            "Description": "The ID of the cluster",
            "Value": {
                "Ref": "Cluster"
            }
        },
        "SrvAddress": {
            "Description": "Connection string for the cluster",
            "Value": {
                "Fn::GetAtt" : [ "Cluster", "SrvAddress" ]
            }
        }
    }

This is a documented attribute for the cluster in the README: https://github.com/mongodb/mongodbatlas-cloudformation-resources/blob/master/cluster/README.md#attributes

When I deploy this to AWS I get an error message: Unable to retrieve SrvAddress attribute for MongoDB::Atlas::Cluster. Rollback requested by user. - and the stack is rolled back.

Is this the correct way to reference attributes for these Mongo Atlas custom resources?

I've also tried "Fn::GetAtt" : [ "Cluster", "Properties.SrvAddress" ] and "Fn::GetAtt" : [ "Cluster", "/properties/SrvAddress" ]

It does seem that I'm using the correct attribute - if I use a different attribute I get a different error "Requested attribute srvAddress does not exist in schema for MongoDB::Atlas::Cluster"

CFN "make schema" command creates JSON schema with empty properties

CloudFormation Template

{
  "definitions": {
    "": {
      "additionalProperties": false
    },
    .
    .
    .
  },
  .,
  .,
  .,
  "properties": {
    "": {},
    .
    .
    .
  },
  .
  .
}

Steps to Reproduce

Generate schema for any API from autogen.

Expected Behavior

The generated schema should not contain empty properties/values ("") as that causes issues when generating corresponding CFN resource.

Actual Behavior

CFN "make schema" command creates JSON schema with empty properties.

SearchIndex: Unable to cast (*string)(0xc0005f04b0) of type *string to map[string]interface{}

Steps to Reproduce

I have just activated MongoDB::Atlas::SearchIndex in AWS with latest version 1.3.0 (i'm already using the cluster, project, user db and other) and now in my CDK code i'm trying to do something like this:

import { CfnSearchIndex } from '@mongodbatlas-awscdk/search-index';

const mySearchIndex = new CfnSearchIndex(this, 'MySearchIndex', {
  analyzer: 'lucene.standard',
  clusterName: 'my-cluster-name',
  collectionName: 'my-collection',
  database: 'my-db',
  mappings: {
    fields: JSON.stringify({
      text_field: {}
    }),
    dynamic: false,
  },
  profile: 'my-profile',
  projectId: this.project.attrId, // Coming from created project resource created before
  searchAnalyzer: 'lucene.standard'
});

Expected Behavior

Should create the index resource.

Actual Behavior

Im seeing the following error when i try to run the CDK code:

Resource handler returned message: "unable to cast (*string)(0xc0005f04b0) of type *string to map[string]interface{}" (RequestTok
en: 753d68e5-1530-ae23-f455-5306ed80fe55, HandlerErrorCode: InvalidRequest)

I already tried to set mappings.fields to other stuff like just the field name and even passing the object itself (not converted to string)... notice that i'm passing string there because of typescript which is telling that fields must be a string.

Additional Context

References

Cloudformation hangs while creating a simple resource.

CloudFormation Template

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "This template creates a Project on the MongoDB Atlas API, this will be billed to your Atlas account.",
  "Parameters": {
    "Profile": {
      "Type": "String",
      "Description": "MongoDB Atlas Profile name"
    },
    "ProjectId": {
      "Type": "String",
      "Description": "MongoDB project Key"
    }
  },
  "Mappings": {},
  "Resources": {
    "CustomDBRoletest": {
      "Type": "MongoDB::Atlas::CustomDBRole",
      "Properties": {
        "ProjectId": {
          "Ref": "ProjectId"
        },
        "Actions": [
          {
            "Action": "FIND",
            "Resources": [
              {
                "Collection": "listingsAndReviews",
                "DB": "sample_airbnb"
              }
            ]
          }
        ],
        "RoleName": "testcreatetest",
        "Profile": {
          "Ref": "Profile"
        }
      }
    }
  },
  "Outputs": {}
}

Steps to Reproduce

Just create the above resource, create relevant secret (with both private public keys), and enable CustomDBRole

Expected Behavior

Should create relevant role in mongo

Actual Behavior

CF hangs, after over an hour + i get internal error
image

Additional Context

i'm trying to create cloudformation stack and create a CustomDbRole
https://github.com/mongodb/mongodbatlas-cloudformation-resources

https://github.com/mongodb/mongodbatlas-cloudformation-resources/blob/master/examples/custom-db-role/custom-db-role.json

This hangs, there are no logs.

  • I've created the required keys in a secret (cfn/atlas/profile/abc) - and the resource does reference the required "abc" as the profile.

the secret looks like this:
{"PrivateKey":"xxx","PublicKey":"yyy"}

i'm not really sure how to debug this, i've also activated the relevant resource
image

serverless name input regex pattern

I tried to create a serverless instance and received an error regarding regex pattern mismatch.
its defined as

    "Name": {
      "type": "string",
      "description": "Human-readable label that identifies the serverless instance.",
      "maxLength": 64,
      "minLength": 1,
      "pattern": "^([a-zA-Z0-9]([a-zA-Z0-9-]){0,21}(?\u003c!-)([\\w]{0,42}))$"
    },

which is oddly specific.
my input looks like my-application-name-env-123 which fails.
after playing around with regexr.com i figured, its because of the negative lookbehind ([a-zA-Z0-9-]){0,21}(?\u003c!-). which basically rejects - characters after the 21st character in the string.

i wonder if this is on purpose as i couldn't find anything in the docs on mongodb.com itself.

Maybe there is some hidden reason behind.

As a user creating names dynamically from applications and cloudformation stacks this forces me to fall back to ([a-zA-Z0-9]{1,64}) and remove all - from my input. works, but is less readable in the atlas interface after creation.

enable adding PrivateEndpoint to a specific security group

Hello,

I would like to report an issue I encountered while attempting to add a MongoDB Atlas private endpoint using CDK. While the endpoint was successfully created, I noticed that there was no option to add the private endpoint to a non-default security group. In my opinion, it would be beneficial to have an option to add security groups in a similar fashion to subnetIds.

To provide some context, I am using the following code to create the private endpoint:

const endpoint = new CfnPrivateEndpoint(scope, 'myPrivateEndpoint', {
            groupId: projectId,
            region: region,
            privateEndpoints: [{
                vpcId: vpc.vpcId,
                subnetIds: vpc.publicSubnets.map(subnet => subnet.subnetId),
            }],
        })

I believe this functionality would be a valuable addition to the current implementation since many companies use non-default security groups and would find this issue a complete blocker for creating private endpoints using CDK.

Thank you for your time and attention to this matter.

Create Database User API Error Response

Im trying to create a User and this cloudformation template passes all cloudformation and internal valiations until the actual POST to https://cloud.mongodb.com/api/atlas error out with

400 (request "MISSING_ATTRIBUTE") The required attribute databaseName was not specified

CloudFormation Template

"mongodbuseratlasiamuser4A29065A": {
   "Type": "MongoDB::Atlas::DatabaseUser",
   "Properties": {
    "Profile": "default",
    "AWSIAMType": "ROLE",
    "Username": {
     "Fn::GetAtt": [
      "lambdaexecutionrole747054AD",
      "Arn"
     ]
    },
    "DatabaseName": "$external",
    "ProjectId": "{{resolve:secretsmanager:arn:aws:secretsmanager:eu-west-1::secret:cfn/atlas/profile/default-kRmT5n:SecretString:ProjectId::}}",
    "Roles": [
     {
      "RoleName": "readWriteAnyDatabase"
     }
    ],
    "Scopes": [
     {
      "Name": "myServerlessInstanceName",
      "Type": "CLUSTER"
     }
    ]
   }
  },

Expected Behavior

create a user or validate possible errors before hitting the API

Actual Behavior

 3:22:44 PM | CREATE_FAILED        | MongoDB::Atlas::DatabaseUser       | mongodb/user/atlas-iam-user (mongodbuseratlasiamuser) Resource handler returned message: "Error getting resource : POST https://cloud.mongodb.com/api/atlas/v1.0/groups/60000/databaseUsers: 400 (request "MISSING_ATTRIBUTE") The required attribute databaseName was not specified." (RequestToken: 2b6d0eb3-1a2a-77ce-f415-5ea8a168ebbb, HandlerErrorCode: InvalidRequest)

Is there any way to debug this on my end?

feat: MongoAtlasBootstrap construct

Idea

The atlas-basic construct is very handy for cluster creation and helps CDK users to jumpstart their mongoDB journey with CDK, however, there are some prerequisites AWS users have to complete before they are allowed to deploy it with atlas-basic or any other high level constructs:

  1. They have to activate the CFN public extension and specify an execution role, which must have relevant trust policy with service principal defined, read the doc for more info.
  2. They have to create a profile secret from ASW Secrets Manager with their Mongo Atlas public key and private key. They are encouraged to deploy this CFN template to generate the secret, which technically and preferably should be completed with AWS CDK as well.

I suggest to have a MongoAtlasBootstrap CDK construct that allows new AWS CDK users to achieve requirements in CDK.

For example:

const app = new cdk.App();
const env = { region: process.env.CDK_DEFAULT_REGION, account: process.env.CDK_DEFAULT_ACCOUNT }
const bootstrap = new cdk.Stack(app, 'mongo-cdk-bootstrap');
const demoStack = new cdk.Stack(app, 'mongo-cdk-demo', { env });


// bootstrap by creating the cfn extension execution role and profile secret.
new MongoAtlasBootstrap(bootstrap, 'mongoCdkBootstrap', { 
  roleName: 'cfn-ext-exec-role-for-mongo',
  secretProfile: 'my-profile',
});

const orgId = demoStack.node.tryGetContext('MONGO_ORG_ID') || process.env.MONGO_ORG_ID;

new AtlasBasic(demoStack, 'atlas-basic', {
  profile: 'pahud-profile',
  clusterProps: { replicationSpecs : replicationSpecs },
  projectProps: { orgId },
  ipAccessListProps,
});

When the user run npx cdk deloy mongo-cdk-bootstrap, the mongo-cdk-bootstrap stack will be deployed and 2 resources will be created:

  1. CFN public extension execution role with well-defined policies and print out the AWS CLI commands to activate relevant mongo atlas extensions.
  2. Create a dummy secret and print out the AWS CLI command prompt so you can update this secret with correct credentials.

The MongoAtlasBootstrap should be deployed in a separate stack only for the first time. I believe this would be very helpful to allow new CDK users to adopt mongo atlas clusters.

Proof of Concept

import { CfnOutput, SecretValue,
    aws_iam as iam,
    aws_secretsmanager as secretsmanager,
} from 'aws-cdk-lib';
import { Construct } from 'constructs';


export class MongoAtlasBootstrapProps {
    /**
     * The IAM role name for CloudFormation Extension Execution.
     * @see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/registry-public.html
     * 
     * @default auto generat the name.
     */
    readonly roleName?: string;
    /**
     * The secret profile name for MongoDB Atlas.
     * @default generate a dummy secret.
     * @see https://github.com/mongodb/mongodbatlas-cloudformation-resources/tree/master#2-configure-your-profile
     */
    readonly secretProfile?: string;
}

/**
 * Generate the CFN extension execution role.
 * @see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/registry-public.html
 */
export class MongoAtlasBootstrap extends Construct {
    readonly role: iam.IRole;
    constructor(scope: Construct, id: string, props?: MongoAtlasBootstrapProps){
        super(scope, id);

        this.role = new iam.Role(this, 'CfnExecRole', {
            assumedBy: new iam.ServicePrincipal('resources.cloudformation.amazonaws.com'),
            roleName: props?.roleName,
            managedPolicies: [
                iam.ManagedPolicy.fromAwsManagedPolicyName('AdministratorAccess'),
            ],
        });

        if(props?.secretProfile) {
            new MongoSecretProfile(this, 'MongoSecretProfile', props?.secretProfile )
        }
        for (let x of ['Cluster', 'Project', 'DatabaseUser', 'ProjectIpAccessList'] ) {
            new CfnOutput(this, `ActivateCmd${x}`, { value: `aws cloudformation activate-type --type-name MongoDB::Atlas::${x} --publisher-id bb989456c78c398a858fef18f2ca1bfc1fbba082 --type RESOURCE --execution-role-arn ${this.role.roleArn}`})
        }
    }
}

export class MongoSecretProfile extends Construct {
    constructor(scope: Construct, id: string, profileName: string) {
        super(scope, id);
        // create a secret
        const secret = new secretsmanager.Secret(this, profileName, {
            secretName: `cfn/atlas/profile/${profileName}`,
            secretStringValue: SecretValue.unsafePlainText('{"PublicKey":"changeMe", "PrivateKey": "changeMe"}'),
        })
        new CfnOutput(this, 'SecretName', { value: secret.secretName });
        new CfnOutput(this, 'UpdateSecretCommand', {
            value: `aws secretsmanager update-secret --secret-id ${secret.secretName}`+ ' --secret-string "{\\"PublicKey\\":\\"${MONGO_ATLAS_PUBLIC_KEY}\\",\\"PrivateKey\\":\\"${MONGO_ATLAS_PRIVATE_KEY}\\"}"'
        })
    }
}

Unable to whitelist AWS Cloudformation

Background

I'm trying to create a Cloudformation stack to stand up a Mongo cluster using registered Mongo private resources. I was able to use the cfn-submit-helper.sh script to build and submit all the private resources without any trouble. The resources register without any problem and appear to be working.

I've also created a CF template describing a cluster deploy. I basically just copied the quickstart template and made some slight modifications:

AWSTemplateFormatVersion: '2010-09-09'
Description: Stack describing a MongoDB cluster for the rest API.
Metadata: <omitted for brevity>
Parameters:
  PublicKey:
    Type: String
    Description: Required. Your MongoDB Cloud Public API Key.
  PrivateKey:
    Type: String
    Description: Required. Your MongoDB Cloud Private API Key.
  OrgId:
    Type: String
    Description: Required. Your MongoDB Cloud Organization Id.
  ProjectName:
    Type: String
    Description: Required. The name of the project.
  ClusterName:
    Type: String
    Description: Required. The name of the cluster as it appears in Atlas. Once the cluster is created,
      its name cannot be changed.
  ClusterInstanceSize:
    Type: String
    Description: Required. The size of the Atlas cluster to spin up. Atlas provides different cluster tiers, each with a default storage capacity and RAM size. The cluster you select is used for all the data-bearing hosts in your cluster tier. See https://docs.atlas.mongodb.com/reference/amazon-aws/#amazon-aws.
    AllowedValues: <omitted for brevity>
  ClusterRegion:
    Type: String
    Description: Required. The AWS Region where the Atlas DB Cluster will run.
    AllowedValues: <omitted for brevity>
  ClusterMongoDBMajorVersion:
    Type: String
    Description: Required. The version of MongoDB that you want the new cluster to run.
    AllowedValues:
    - "3.6"
    - "4.0"
    - "4.2"
    - "4.4"
  DatabaseUserRoleDatabaseName:
    Type: String
    Description: Database User Role Database Name
Resources:
  AtlasApiKeySecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Sub "${AWS::StackName}-ApiKey-Secret"
      Description: MongoDB Atlas API Key
      #
      # { "PublicKey" : "xxx", "PrivateKey", "yyy", "OrgId": "zzz" }
      #
      SecretString: !Join ['', ['{"PublicKey":"', !Ref "PublicKey",'", "PrivateKey":"', !Ref "PrivateKey", '", "OrgId":"', !Ref "OrgId", '"}'] ]
      Tags:
      - Key: mongodb-atlas-quickstart-stack
        Value: !Sub "${AWS::StackName}"
  AtlasIAMRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Principal:
            AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
          Action: 'sts:AssumeRole'
  AtlasProject:
    Type: MongoDB::Atlas::Project
    DependsOn: AtlasIAMRole
    Properties:
      OrgId: !Ref "OrgId"
      ApiKeys:
        PublicKey:  !Ref "PublicKey"
        PrivateKey: !Ref "PrivateKey"
      Name: !Ref "ProjectName"
  AtlasProjectIPAccessList:
    Type: MongoDB::Atlas::ProjectIpAccessList
    DependsOn: AtlasProject
    Properties:
      ProjectId: !Ref "AtlasProject"
      ApiKeys:
        PublicKey:  !Ref "PublicKey"
        PrivateKey: !Ref "PrivateKey"
      AccessList:
      - IPAddress: "0.0.0.0/1"
        Comment: "Testing open all ips"
  AtlasCluster:
    Type: MongoDB::Atlas::Cluster
    DependsOn: AtlasProject
    Properties:
      ApiKeys:
        PublicKey:  !Ref "PublicKey"
        PrivateKey: !Ref "PrivateKey"
      ProjectId: !Ref "AtlasProject"
      Name: !Ref "ClusterName"
      MongoDBMajorVersion: !Ref "ClusterMongoDBMajorVersion"
      ReplicationFactor: 3
      NumShards: 1
      ProviderSettings:
        ProviderName: "AWS"
        InstanceSizeName: !Ref "ClusterInstanceSize"
        RegionName: !Ref "ClusterRegion"
  AtlasDatabaseUser:
    Type: MongoDB::Atlas::DatabaseUser
    DependsOn: AtlasCluster
    Properties:
      ProjectId: !Ref "AtlasProject"
      ApiKeys:
        PublicKey:  !Ref "PublicKey"
        PrivateKey: !Ref "PrivateKey"
      Username: !GetAtt "AtlasIAMRole.Arn"
      DatabaseName: "$external"
      AWSIAMType: "ROLE"
      Roles:
      - RoleName: "readWrite"
        DatabaseName: !Ref "DatabaseUserRoleDatabaseName"
      Scopes:
      - Name: !Sub "${AWS::StackName}"
        Type: "CLUSTER"
Outputs: <omitted for brevity>

The problem

The problem is that when I run my Cloudformation template, the AtlasProject resource fails to create with the following error:

{
    "level": "debug",
    "msg": "Create - error: POST https://cloud.mongodb.com/api/atlas/v1.0/groups: 403 (request \"Forbidden\") IP address 18.118.114.43 is not allowed to access this resource.",
    "time": "2021-06-12T14:39:15Z"
}

18.118.114.43 is the address of the Amazon Cloudformation server building the stack. Obviously, I can't whitelist this address in my Atlas organization's IP access lists because I don't know it before the stack is run.

I made the following attempts to fix the problem:

  • I added 18.118.114.43 to my Atlas organization's whitelist. This failed because the IP address changed each time the stack was run (i.e., 52.15.211.239 and 18.191.254.107)
  • I attempted to add 0.0.0.0/0 to my Atlas organization's whitelist. However, that failed with the following error: The address 0.0.0.0/0 cannot be added to access lists.

At this point, it seems like the only way to fix this problem is to whitelist Amazon's block of IP addresses for my AWS region if I want the CF templates to work. This is not only impractical, it's insecure.

Expected behavior

After I've built and registered the MongoDB Cloudformation resources in my AWS organization's private registry, I should be able to use them in CF templates without whitelisting all of Amazon's IP addresses.

The documentation should explain how to get around this issue. Right now all it says is, "You should allow API access for the IP address from which the acceptance test runs", which isn't very meaningful because that's not the same IP address that CF will be using to provision the stack.

Any help is much appreciated.

AWS default profile in config files

Steps to Reproduce

I am trying to clone and run this repository to execute the cfn-submit-helper.sh file to register all the resources in our aws account. When I run this repo in my Azure DevOps pipeline I get an error because in all the config files there is a default profile set to "711489243244_AdministratorAccess" and I can't see any way of sending a different profile as a parameter to the .sh file.
I think there should be a parameter to the script-file and there maybe shouldn't be any hard-coded default profile checked in to the repo?

Reference

mongodbatlas-cloudformation-resources/cfn-resources/cfn-submit-helper.config
export CFN_SUBMIT_LOG_LEVEL=info
export CFN_SUBMIT_CFN_FLAGS="--verbose --set-default"
export AWS_DEFAULT_PROFILE=711489243244_AdministratorAccess

Can't set up Auto Scaling with Cluster resource

Hi,

I'm having an issue creating an Atlas cluster with autoscaling enabled. Given the below resource definition, I get a 400 error with the message Compute auto-scaling min instance size required.. However, if I specify AutoScaling.Compute.MaxInstanceSize and/or AutoScaling.Compute.MinInstanceSize in my resource definition, I get another error saying Invalid attribute minInstanceSize specified. Is Cluster Tier scaling supported, and if so, do y'all have a working example? Thanks in advance.

Code snippet

  "OrganizerServicesTestInstanceAtlasClsutertestinstance543CDFC6": {
   "Type": "MongoDB::Atlas::Cluster",
   "Properties": {
    "ApiKeys": {
     "PublicKey": "<REDACTED>",
     "PrivateKey": "<REDACTED>"
    },
    "ProjectId": {
     "Ref": "OrganizerServicesAtlasProjectAtlasProjectsandboxE89594CB"
    },
    "Name": "test-instance",
    "MongoDBMajorVersion": "5.0",
    "ProviderSettings": {
     "BackingProviderName": "AWS",
     "InstanceSizeName": "M10",
     "RegionName": "us-east-1",
     "AutoScaling": {
      "Compute": {
       "Enabled": true,
       "ScaleDownEnabled": true,
       "MinInstanceSize": "M10",
       "MaxInstanceSize": "M50"
      }
     }
    },
    "AutoScaling": {
     "Compute": {
      "Enabled": true,
      "ScaleDownEnabled": true
     }
    }
   },
   "DependsOn": [
    "OrganizerServicesAtlasProjectAtlasProjectsandboxE89594CB"
   ],
   "Metadata": {
    "aws:cdk:path": "OrganizerServicesInfrastructureStack-sandbox/OrganizerServicesTestInstance/AtlasClsuter-test-instance"
   }
  },

Expected Behavior

200 response, cluster created with cluster tier scaling enabled.

Actual behavior

400, Compute auto-scaling min instance size required. if AutoScaling.Compute.MinInstanceSize is not set,
400, Invalid attribute minInstanceSize specified. if AutoScaling.Compute.MinInstanceSize is set.

Build not working with cloudformation-cli-go-plugin v0.1.5

I am trying to build the resource using make command on my machine and it fails with following message.

cfn generate
Change message for Go plugin v0.1.3:
Generated models no longer use the types exported in the encoding package.
Your model's fields have been regenerated using standard pointer types (*string, *int, etc) as used in the AWS Go SDK.
The AWS SDK has helper functions that you can use to get and set your model's values.

Make the following changes to your handler code as needed:

* Replace `encoding.New{Type}` with `aws.{Type}`
* Replace `model.{field}.Value()` with `aws.{Type}Value(model.{field})`

Where {Type} is either String, Bool, Int, or Float64 and {field} is any field within your generated model.

Generated files for MongoDB::Atlas::NetworkPeering
env GOOS=linux go build -ldflags="-s -w" -tags="logging callback metrics scheduler" -o bin/handler cmd/main.go
# github.com/mongodb/mongodbatlas-cloudformation-resources/network-peering/cmd/resource
cmd/resource/resource.go:14:73: currentModel.ApiKeys.PublicKey.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:14:115: currentModel.ApiKeys.PrivateKey.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:20:38: currentModel.ProjectId.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:22:41: currentModel.ContainerId.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:25:43: currentModel.AccepterRegionName.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:29:43: currentModel.AwsAccountId.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:33:44: currentModel.RouteTableCidrBlock.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:37:29: currentModel.VpcId.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:41:43: currentModel.ProviderName.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:57:18: cannot use encoding.NewString(peerResponse.ID) (type *encoding.String) as type *string in assignment
cmd/resource/resource.go:57:18: too many errors
make: *** [Makefile:5: build] Error 2

Building MongoDB::Atlas::DatabaseUser and MongoDB::Atlas::EncryptionAtRest fails

Environment:

> pip show cloudformation-cli

Name: cloudformation-cli
Version: 0.1.3
Summary: UNKNOWN
Home-page: https://github.com/aws-cloudformation/aws-cloudformation-rpdk/
Author: Amazon Web Services
Author-email: [email protected]
License: Apache License 2.0
Location: /Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages
Requires: jsonschema, Werkzeug, colorama, requests, hypothesis, pytest, PyYAML, boto3, Jinja2
Required-by: cloudformation-cli-go-plugin

and ...

pip show cloudformation-cli-go-plugin

Name: cloudformation-cli-go-plugin
Version: 0.1.2
Summary: UNKNOWN
Home-page: https://github.com/aws-cloudformation/cloudformation-cli-go-plugin/
Author: Amazon Web Services
Author-email: [email protected]
License: Apache License 2.0
Location: /Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages
Requires: cloudformation-cli
Required-by: 

Error for MongoDB::Atlas::EncryptionAtRest

When building MongoDB::Atlas::EncryptionAtRest with make:

>    make

cfn generate
resource/model.go
main.go
=== Unhandled exception ===
Please report this issue to the team.
Issue tracker: https://github.com/aws-cloudformation/aws-cloudformation-rpdk/issues
Please include the log file 'rpdk.log'
make: *** [build] Error 127

Content of rpdk.log:

[2020-05-14T21:13:54Z] DEBUG    - Logging set up successfully
[2020-05-14T21:13:54Z] DEBUG    - Running generate: Namespace(command=<function generate at 0x10dd3cf80>, subparser_name='generate', verbose=0, version=False)
[2020-05-14T21:13:54Z] DEBUG    - Root directory: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest
[2020-05-14T21:13:54Z] DEBUG    - Loading project file '/Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/.rpdk-config'
[2020-05-14T21:13:54Z] INFO     - Validating your resource specification...
[2020-05-14T21:13:54Z] DEBUG    - Resolved '#/definitions/AwsKms' to 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/mongodb-atlas-encryptionatrest.json#/definitions/AwsKms'
[2020-05-14T21:13:54Z] DEBUG    - Parsed 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/mongodb-atlas-encryptionatrest.json#/definitions/AwsKms' to '(<BASE>, 'definitions', 'AwsKms')'
[2020-05-14T21:13:54Z] DEBUG    - Edge from '(<BASE>, 'properties', 'AwsKms')' to '(<BASE>, 'definitions', 'AwsKms')'
[2020-05-14T21:13:54Z] DEBUG    - Resolved '#/definitions/apiKeyDefinition' to 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/mongodb-atlas-encryptionatrest.json#/definitions/apiKeyDefinition'
[2020-05-14T21:13:54Z] DEBUG    - Parsed 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/mongodb-atlas-encryptionatrest.json#/definitions/apiKeyDefinition' to '(<BASE>, 'definitions', 'apiKeyDefinition')'
[2020-05-14T21:13:54Z] DEBUG    - Edge from '(<BASE>, 'properties', 'ApiKeys')' to '(<BASE>, 'definitions', 'apiKeyDefinition')'
[2020-05-14T21:13:54Z] DEBUG    - Rewriting refs in '<BASE>' (file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/mongodb-atlas-encryptionatrest.json)
[2020-05-14T21:13:54Z] DEBUG    -   '#/definitions/AwsKms' -> '#/definitions/AwsKms'
[2020-05-14T21:13:54Z] DEBUG    -   '#/definitions/apiKeyDefinition' -> '#/definitions/apiKeyDefinition'
[2020-05-14T21:13:54Z] DEBUG    - Writing Resource Role CloudFormation template: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/resource-role.yaml
[2020-05-14T21:13:54Z] DEBUG    - Overwriting '/Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/resource-role.yaml'
[2020-05-14T21:13:54Z] DEBUG    - Generate started
[2020-05-14T21:13:54Z] DEBUG    - Writing Types
[2020-05-14T21:13:54Z] DEBUG    - Overwriting '/Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/cmd/resource/model.go'
[2020-05-14T21:13:54Z] DEBUG    - Writing project: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/cmd/main.go
[2020-05-14T21:13:54Z] DEBUG    - Overwriting '/Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/cmd/main.go'
[2020-05-14T21:13:55Z] DEBUG    - Removing generated docs: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/docs
[2020-05-14T21:13:55Z] DEBUG    - Writing generated docs
[2020-05-14T21:13:55Z] DEBUG    - Writing docs awskms.md: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/docs/awskms.md
[2020-05-14T21:13:55Z] DEBUG    - Writing docs apikeys.md: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/encryption-at-rest/docs/apikeys.md
[2020-05-14T21:13:55Z] DEBUG    - Finished documenting nested properties
[2020-05-14T21:13:55Z] DEBUG    - Unhandled exception
Traceback (most recent call last):
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/cli.py", line 98, in main
    args.command(args)
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/generate.py", line 16, in generate
    project.generate_docs()
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/project.py", line 373, in generate_docs
    getatt = self._get_docs_gettable_atts(docs_schema)
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/project.py", line 413, in _get_docs_gettable_atts
    for prop in docs_schema.get("readOnlyProperties", [])
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/project.py", line 413, in <listcomp>
    for prop in docs_schema.get("readOnlyProperties", [])
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/project.py", line 402, in _get_property_description
    name = path[-1]
IndexError: tuple index out of range

Error for MongoDB::Atlas::DatabaseUser

When building MongoDB::Atlas::DatabaseUser with make:

>    make

cfn generate
resource/model.go
main.go
=== Unhandled exception ===
Please report this issue to the team.
Issue tracker: https://github.com/aws-cloudformation/aws-cloudformation-rpdk/issues
Please include the log file 'rpdk.log'
make: *** [build] Error 127

Content of rpdk.log:

[2020-05-14T21:22:23Z] DEBUG    - Logging set up successfully
[2020-05-14T21:22:23Z] DEBUG    - Running generate: Namespace(command=<function generate at 0x101c3cf80>, subparser_name='generate', verbose=0, version=False)
[2020-05-14T21:22:23Z] DEBUG    - Root directory: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user
[2020-05-14T21:22:23Z] DEBUG    - Loading project file '/Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/.rpdk-config'
[2020-05-14T21:22:23Z] INFO     - Validating your resource specification...
[2020-05-14T21:22:23Z] DEBUG    - Resolved '#/definitions/labelDefinition' to 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/mongodb-atlas-databaseuser.json#/definitions/labelDefinition'
[2020-05-14T21:22:23Z] DEBUG    - Parsed 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/mongodb-atlas-databaseuser.json#/definitions/labelDefinition' to '(<BASE>, 'definitions', 'labelDefinition')'
[2020-05-14T21:22:23Z] DEBUG    - Edge from '(<BASE>, 'properties', 'Labels', 'items')' to '(<BASE>, 'definitions', 'labelDefinition')'
[2020-05-14T21:22:23Z] DEBUG    - Resolved '#/definitions/roleDefinition' to 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/mongodb-atlas-databaseuser.json#/definitions/roleDefinition'
[2020-05-14T21:22:23Z] DEBUG    - Parsed 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/mongodb-atlas-databaseuser.json#/definitions/roleDefinition' to '(<BASE>, 'definitions', 'roleDefinition')'
[2020-05-14T21:22:23Z] DEBUG    - Edge from '(<BASE>, 'properties', 'Roles', 'items')' to '(<BASE>, 'definitions', 'roleDefinition')'
[2020-05-14T21:22:23Z] DEBUG    - Resolved '#/definitions/apiKeyDefinition' to 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/mongodb-atlas-databaseuser.json#/definitions/apiKeyDefinition'
[2020-05-14T21:22:23Z] DEBUG    - Parsed 'file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/mongodb-atlas-databaseuser.json#/definitions/apiKeyDefinition' to '(<BASE>, 'definitions', 'apiKeyDefinition')'
[2020-05-14T21:22:23Z] DEBUG    - Edge from '(<BASE>, 'properties', 'ApiKeys')' to '(<BASE>, 'definitions', 'apiKeyDefinition')'
[2020-05-14T21:22:23Z] DEBUG    - Rewriting refs in '<BASE>' (file:///Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/mongodb-atlas-databaseuser.json)
[2020-05-14T21:22:23Z] DEBUG    -   '#/definitions/labelDefinition' -> '#/definitions/labelDefinition'
[2020-05-14T21:22:23Z] DEBUG    -   '#/definitions/roleDefinition' -> '#/definitions/roleDefinition'
[2020-05-14T21:22:23Z] DEBUG    -   '#/definitions/apiKeyDefinition' -> '#/definitions/apiKeyDefinition'
[2020-05-14T21:22:23Z] DEBUG    - Writing Resource Role CloudFormation template: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/resource-role.yaml
[2020-05-14T21:22:23Z] DEBUG    - Overwriting '/Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/resource-role.yaml'
[2020-05-14T21:22:23Z] DEBUG    - Generate started
[2020-05-14T21:22:23Z] DEBUG    - Writing Typesi
[2020-05-14T21:22:23Z] DEBUG    - Overwriting '/Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/cmd/resource/model.go'
[2020-05-14T21:22:23Z] DEBUG    - Writing project: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/cmd/main.go
[2020-05-14T21:22:23Z] DEBUG    - Overwriting '/Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/cmd/main.go'
[2020-05-14T21:22:23Z] DEBUG    - Removing generated docs: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/docs
[2020-05-14T21:22:23Z] DEBUG    - Writing generated docs
[2020-05-14T21:22:23Z] DEBUG    - Writing docs labels.md: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/docs/labels.md
[2020-05-14T21:22:23Z] DEBUG    - Writing docs roles.md: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/docs/roles.md
[2020-05-14T21:22:23Z] DEBUG    - Writing docs apikeys.md: /Users/azec/dev/go-workspace/src/mongodbatlas-cloudformation-resources-v013/database-user/docs/apikeys.md
[2020-05-14T21:22:23Z] DEBUG    - Finished documenting nested properties
[2020-05-14T21:22:23Z] DEBUG    - Unhandled exception
Traceback (most recent call last):
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/cli.py", line 98, in main
    args.command(args)
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/generate.py", line 16, in generate
    project.generate_docs()
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/project.py", line 373, in generate_docs
    getatt = self._get_docs_gettable_atts(docs_schema)
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/project.py", line 413, in _get_docs_gettable_atts
    for prop in docs_schema.get("readOnlyProperties", [])
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/project.py", line 413, in <listcomp>
    for prop in docs_schema.get("readOnlyProperties", [])
  File "/Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages/rpdk/core/project.py", line 402, in _get_property_description
    name = path[-1]
IndexError: tuple index out of range

SUMMARY: In both instances , issue is presence of empty readOnlyProperties array in JSON resource definition schema. When removed , build passes fine for both resources.

"readOnlyProperties": [
        ""
    ],

Problematic lines:

  1. MongoDB::Atlas::EncryptionAtRest : https://github.com/mongodb/mongodbatlas-cloudformation-resources/blob/master/encryption-at-rest/mongodb-atlas-encryptionatrest.json#L63
  2. MongoDB::Atlas::DatabaseUser : https://github.com/mongodb/mongodbatlas-cloudformation-resources/blob/master/database-user/mongodb-atlas-databaseuser.json#L103-L105

ProjectId field missing from CfnAlertConfigurationProps

Hello,

I would like to report an issue I have encountered trying to set up alerts through CDK. ProjectId is a required field in AlertConfiguration however it is not part of CfnAlertConfigurationProps and therefore it is impossible to even deploy an AlertConfiguration through CDK.

A workaround is to use CfnResource provided by AWS in aws-cdk-lib however I don't think this is a desired solution.

Thank you very much for the time looking into this matter.

Kind Regards

Properties validation failed for MongoDB Atlas Private Endpoint with message: extraneous key [ApiKeys] is not permitted

Description

I am using AWS CDK version 2.66.1 to create a MongoDB Atlas Private Endpoint using the @mongodbatlas-awscdk/private-endpoint library. However, the creation failed with the following error message:

CREATE_FAILED | MongoDB::Atlas::PrivateEndpoint | PrivateEndpoint
Properties validation failed for resource PrivateEndpoint with message:
#: extraneous key [ApiKeys] is not permitted

CDK Code Snippet

import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { IpAddresses, Port, SecurityGroup, Vpc } from 'aws-cdk-lib/aws-ec2';
import { Secret } from 'aws-cdk-lib/aws-secretsmanager';
import { CfnPrivateEndpoint } from '@mongodbatlas-awscdk/private-endpoint';

export interface MongoTestStackProps extends StackProps {
    readonly mongoProjectId: string;
}

export class MongoTestStack extends Stack {
    constructor(scope: Construct, id: string, props: MongoTestStackProps) {
        super(scope, id, props);

        const { account, region } = this;
        const { mongoProjectId } = props;

        const ipAddresses = IpAddresses.cidr('10.0.0.0/22');

        const vpc = new Vpc(this, 'vpc', {
            maxAzs: 2,
            ipAddresses,
            enableDnsSupport: true,
            enableDnsHostnames: true,
            natGateways: 1,
        });

        const securityGroup = new SecurityGroup(this, 'AtlasPrivateLinkSecurityGroup', {
            description: 'atlas privatelink',
            vpc,
        });

        securityGroup.addIngressRule(securityGroup, Port.allTraffic());

        const secrets = Secret.fromSecretCompleteArn(
            this,
            'AtlasApiKeys',
            `arn:aws:secretsmanager:${region}:${account}:secret:****************`,
        );

        const publicKey = secrets.secretValueFromJson('publicKey').unsafeUnwrap();
        const privateKey = secrets.secretValueFromJson('privateKey').unsafeUnwrap();

        const vpcId = vpc.vpcId;
        const subnetIds = vpc.privateSubnets.map(({ subnetId }) => subnetId);

        new CfnPrivateEndpoint(this, 'PrivateEndpoint', {
            groupId: mongoProjectId,
            region,
            privateEndpoints: [
                {
                    vpcId,
                    subnetIds,
                },
            ],
            apiKeys: {
                publicKey,
                privateKey,
            },
        });
    }
}

Steps to Reproduce

  1. Create a new AWS CDK project.
  2. Add a MongoDB Atlas Private Endpoint to the project using the @mongodbatlas-awscdk/private-endpoint library.
  3. Specify the required properties for the Private Endpoint, including the ApiKeys property.
  4. Run cdk deploy to deploy the stack.

Expected Behavior

The MongoDB Atlas Private Endpoint should be created successfully.

Actual Behavior

The creation fails with the error message "Properties validation failed for resource PrivateEndpoint with message: #: extraneous key [ApiKeys] is not permitted".

Additional Context

  • AWS CDK version 2.66.1
  • @mongodbatlas-awscdk/private-endpoint version 1.0.1
  • Node.js version 16.19.1
  • macOS version 12.6.3

Importing Resources Fails to Create Change Set

---
AWSTemplateFormatVersion: '2010-09-09'
Description: MongoDB Atlas AWS CloudFormation Quickstart with VPC Peering. (qs-1rkorhefe)
Metadata:
  cfn-lint: { config: { ignore_checks: [ W9002, W9003, W9006, E3001, E1010 ] } }
  AWS::CloudFormation::Interface:
    ParameterGroups:
    - Label:
        default: Network Configuration
      Parameters:
      - VPC
      - RouteTableCIDRBlock
      - VPCRegion
      - AtlasCidrBlock
    - Label:
        default: MongoDB Atlas ApiKey Configuration
      Parameters:
      - Profile
    - Label:
        default: MongoDB Atlas Configuration
      Parameters:
      - ProjectName
      - ClusterName
      - ClusterRegion
      - ClusterInstanceSize
      - DatabaseUserRoleDatabaseName
      - DatabaseUserName
    ParameterLabels:
      VPC:
        default: VPC-id to peer with Atlas cluster
      AtlasCidrBlock:
        default: IP addresses expressed in Classless Inter-Domain Routing (CIDR) notation.
      RouteTableCIDRBlock:
        default: CIDR block for app to use in VPC Peering.
      Profile:
        default: A secret with name cfn/atlas/profile/{Profile}
      OrgId:
        default: MongoDB Atlas API OrgId
      ProjectName:
        default: Name of new Atlas Project
      ClusterName:
        default: Name of new cluster
      ClusterRegion:
        default: The AWS Region for Atlas Cluster
      ClusterInstanceSize:
        default: MongoDB Atlas Instance Size
      DatabaseUserRoleDatabaseName:
        default: MongoDB Atlas Database User Role Database Name
      VPCRegion:
        default: The AWS Region for VPC
      DatabaseUserName:
        default: MongoDB Atlas Database User Name
Parameters:
  Profile:
    Description: "A secret with name cfn/atlas/profile/{Profile}"
    Type: String
    Default: "default"
  OrgId:
    Description: "Your MongoDB Cloud Organization Id"
    Type: String
    Default: "OrgId"
  ProjectName:
    Description: "The name of the project."
    Type: String
    Default: "aws-quickstart-vpc"
  ClusterName:
    Description: Name of the cluster as it appears in Atlas. Once the cluster is created,
      its name cannot be changed.
    Type: String
    Default: "Cluster-1"
  ClusterInstanceSize:
    Default: "M10"
    Description: Atlas provides different cluster tiers, each with a default storage capacity and RAM size. The cluster you select is used for all the data-bearing hosts in your cluster tier. See https://docs.atlas.mongodb.com/reference/amazon-aws/#amazon-aws.
    Type: String
    AllowedValues:
    - "M10"
    - "M20"
    - "M30"
    - "M40"
    - "R40"
    - "M40_NVME"
    - "M50"
    - "R50"
    - "M50_NVME"
    - "M60"
    - "R60"
    - "M60_NVME"
    - "M80"
    - "R80"
    - "M80_NVME"
    - "M140"
    - "M200"
    - "R200"
    - "M200_NVME"
    - "M300"
    - "R300"
    - "R400"
    - "M400_NVME"
    - "R700"
  ClusterRegion:
    Default: "US_EAST_1"
    Description: AWS Region where the Atlas database runs.
    Type: String
    AllowedValues:
      - "US_EAST_1"
      - "US_EAST_2"
      - "CA_CENTRAL_1"
      - "US_WEST_1"
      - "US_WEST_2"
      - "SA_EAST_1"
      - "AP_SOUTH_1"
      - "AP_EAST_1"
      - "AP_SOUTHEAST_1"
      - "AP_SOUTHEAST_2"
      - "AP_SOUTHEAST_3"
      - "AP_NORTHEAST_1"
      - "AP_NORTHEAST_2"
      - "AP_NORTHEAST_3"
      - "EU_CENTRAL_1"
      - "EU_WEST_1"
      - "EU_NORTH_1"
      - "EU_WEST_2"
      - "EU_WEST_3"
      - "EU_SOUTH_1"
      - "ME_SOUTH_1"
      - "AF_SOUTH_1"
  ClusterMongoDBMajorVersion:
    Description: The version of MongoDB
    Type: String
    Default: "5.0"
    AllowedValues:
      - "4.4"
      - "5.0"
      - "6.0"
  DatabaseUserRoleDatabaseName:
    Description: Database Name
    Type: String
    Default: "admin"
  VPC:
    Type: AWS::EC2::VPC::Id
    Description: VPC-ID of your existing Virtual Private Cloud (VPC) which you wish to peer to your new MongoDB Atlas cluster. This is the VPC that your application uses usually.
  RouteTableCIDRBlock:
    Type: String
    Description: CIDR block to use for your VPC Peering.
  VPCRegion:
    Default: 'us-east-1'
    Description: AWS Region where the Quick Start S3 bucket (QSS3BucketName) is hosted. If you use your own bucket, you must specify this value.
    Type: String
  DatabaseUserName:
    Description: MongoDB Atlas Database User  Name.
    Type: String
    Default: "testUser"
  AtlasCidrBlock:
    Type: String
    Description: IP addresses expressed in Classless Inter-Domain Routing (CIDR) notation that MongoDB Cloud uses for the network peering containers in your project.
Resources:
  AtlasProject:
    Type: MongoDB::Atlas::Project
    DeletionPolicy: Retain
    Properties:
      OrgId: !Ref "OrgId"
      Profile:  !Ref "Profile"
      Name: !Ref "ProjectName"
  AtlasProjectIPAccessList:
    Type: MongoDB::Atlas::ProjectIpAccessList
    DeletionPolicy: Retain
    Properties:
      ProjectId: !GetAtt "AtlasProject.Id"
      Profile:  !Ref "Profile"
      AccessList:
        - IPAddress: "0.0.0.0/0"
          Comment: "Testing open all ips"
  AtlasNetworkPeering:
    Type: MongoDB::Atlas::NetworkPeering
    DeletionPolicy: Retain
    DependsOn: NetworkContainer
    Properties:
      ProjectId: !GetAtt "AtlasProject.Id"
      Profile:  !Ref "Profile"
      AccepterRegionName: !Ref "VPCRegion"
      AwsAccountId: !Sub "${AWS::AccountId}"
      RouteTableCIDRBlock: !Ref "RouteTableCIDRBlock"
      VpcId: !Ref "VPC"
      ContainerId: !GetAtt "NetworkContainer.Id"
  AtlasCluster:
    Type: MongoDB::Atlas::Cluster
    DeletionPolicy: Retain
    Properties:
      Profile:  !Ref "Profile"
      ProjectId: !GetAtt "AtlasProject.Id"
      Name: !Ref "ClusterName"
      MongoDBMajorVersion: !Ref "ClusterMongoDBMajorVersion"
      ClusterType: "REPLICASET"
      ReplicationSpecs:
        - NumShards: '1'
          AdvancedRegionConfigs:
            - AutoScaling:
                DiskGB:
                  Enabled: 'true'
                Compute:
                  Enabled: 'false'
                  ScaleDownEnabled: 'false'
              AnalyticsSpecs:
                EbsVolumeType: STANDARD
                InstanceSize: !Ref "ClusterInstanceSize"
                NodeCount: '3'
              ElectableSpecs:
                EbsVolumeType: STANDARD
                InstanceSize: !Ref "ClusterInstanceSize"
                NodeCount: '3'
              ReadOnlySpecs:
                EbsVolumeType: STANDARD
                InstanceSize: !Ref "ClusterInstanceSize"
                NodeCount: '3'
              Priority: '7'
              RegionName: !Ref ClusterRegion
  NetworkContainer:
    Type: MongoDB::Atlas::NetworkContainer
    DeletionPolicy: Retain
    DependsOn: AtlasCluster
    Properties:
      Profile:  !Ref "Profile"
      AtlasCidrBlock: !Ref AtlasCidrBlock
      RegionName: !Ref ClusterRegion
      ProjectId: !GetAtt "AtlasProject.Id"

Steps to Reproduce

Attempt to create a change set with the above CloudFormation template with valid and appropriate parameters and properties like so:

Parameters

[
  {
    "ParameterKey": "OrgId",
    "ParameterValue": "REDACTED"
  },
  {
    "ParameterKey": "ProjectName",
    "ParameterValue": "REDACTED"
  },
  {
    "ParameterKey": "ClusterName",
    "ParameterValue": "cluster-1"
  },
  {
    "ParameterKey": "ClusterInstanceSize",
    "ParameterValue": "M10"
  },
  {
    "ParameterKey": "ClusterRegion",
    "ParameterValue": "AP_SOUTHEAST_2"
  },
  {
    "ParameterKey": "ClusterMongoDBMajorVersion",
    "ParameterValue": "4.4"
  },
  {
    "ParameterKey": "DatabaseUserRoleDatabaseName",
    "ParameterValue": ""
  },
  {
    "ParameterKey": "VPC",
    "ParameterValue": "REDACTED"
  },
  {
    "ParameterKey": "RouteTableCIDRBlock",
    "ParameterValue": "10.50.0.0/16"
  },
  {
    "ParameterKey": "VPCRegion",
    "ParameterValue": "ap-southeast-2"
  },
  {
    "ParameterKey": "AtlasCidrBlock",
    "ParameterValue": "172.31.0.0/21"
  },
  {
    "ParameterKey": "Profile",
    "ParameterValue": "MongoDB"
  }
]

Properties

[
  {
    "ResourceType": "MongoDB::Atlas::Project",
    "LogicalResourceId": "AtlasProject",
    "ResourceIdentifier": {
      "Profile": "MongoDB",
      "Id": "REDACTED"
    }
  },
  {
    "ResourceType": "MongoDB::Atlas::NetworkPeering",
    "LogicalResourceId": "AtlasNetworkPeering",
    "ResourceIdentifier": {
      "Profile": "MongoDB",
      "Id": "REDACTED",
      "ProjectId": "REDACTED"
    }
  },
  {
    "ResourceType": "MongoDB::Atlas::Cluster",
    "LogicalResourceId": "AtlasCluster",
    "ResourceIdentifier": {
      "Profile": "MongoDB",
      "ProjectId": "REDACTED",
      "Name": "cluster-1"
    }
  },
  {
    "ResourceType": "MongoDB::Atlas::NetworkContainer",
    "LogicalResourceId": "NetworkContainer",
    "ResourceIdentifier": {
      "Profile": "MongoDB",
      "Id": "REDACTED",
      "ProjectId": "REDACTED"
    }
  },
  {
    "ResourceType": "MongoDB::Atlas::ProjectIpAccessList",
    "LogicalResourceId": "AtlasProjectIPAccessList",
    "ResourceIdentifier": {
      "Profile": "MongoDB",
      "ProjectId": "REDACTED"
    }
  }
]

Additionally, a secret exists in the appropriate region that is appropriately formed with the appropriate name cfn/atlas/profile/MongoDB.

Expected Behavior

A change set is created that can be executed to import the Atlas resources.

Actual Behavior

The change set remains in the CREATE_IN_PROGRESS state before failing after an hour or so.

Additional Context

This has been an issue affecting a production application with a paid Atlas subscription for 4 or so months now.

References

#204

Project Creation Fails on Cloudformation, while its successfully created on MongoDbAtlas

I used the latest build from releases and tried by applying the example stack for project.
It successfully creates the project on MongoDb Atlas, but on cloudformation it shows create failed and goes into rollback. (also deletion during the rollback doesn't delete the project on MongoDb atlas).
The reason of failure as returned by cloudformation is:

Unable to complete request: error creating project: POST https://cloud.mongodb.com/api/atlas/v1.0/groups: 409 (request "Conflict") A group with name "cloudformation_resource_demo" already exists.

Unable to update ProjectIPAccessList IPs

Hi,

I stumbled upon a bug when updating Project IP Access List using Cloudformation or CDK to be precise.
I cannot change any entries in accessList.

To reproduce:

new CfnProjectIpAccessList(scope, `${stackName}AtlasProjectIpAccessList`, {
            projectId,
            accessList: [
                {
                    ipAddress: '0.0.0.0/0',
                    comment: 'any',
                },
            ],
        })

when changed to

new CfnProjectIpAccessList(scope, `${stackName}AtlasProjectIpAccessList`, {
            projectId,
            accessList: [
                {
                    ipAddress: '10.0.0.0/8',
                    comment: 'vpc',
                },
            ],
        })

I receive an error: Resource handler returned message: "The entry to update is not in the access list".

Similarly, when changed to

new CfnProjectIpAccessList(scope, `${stackName}AtlasProjectIpAccessList`, {
            projectId,
            accessList: [
                {
                    ipAddress: '0.0.0.0/0',
                    comment: 'any',
                },
                {
                    ipAddress: '10.0.0.0/8',
                    comment: 'vpc',
                },
            ],
        })

I receive an error: Resource handler returned message: "Error deleting the resource: DELETE https://cloud.mongodb.com/api/atlas/v1.0/groups/xxx/accessList/10.0.0.0%2F8: 404 (request "ATLAS_NETWORK_PERMISSION_ENTRY_NOT_FOUND") IP Address 10.0.0.0/8 not on Atlas access list for group xxx..

I think this is unintended behaviour and prevents updates to ProjectIPAccessList resource.

Import Resources Failing

Hello,

I've been experiencing an issue when attempting to create a CloudFormation stack from existing resources where I am unable to import any resources.

Here is my template which is a slightly modified version of this:

AWSTemplateFormatVersion: '2010-09-09'
Description: MongoDB Atlas AWS CloudFormation Quickstart with VPC Peering. (qs-1rkorhefe)
Metadata:
  cfn-lint: { config: { ignore_checks: [ W9002, W9003, W9006, E3001, E1010 ] } }
  AWS::CloudFormation::Interface:
    ParameterGroups:
    - Label:
        default: Network Configuration
      Parameters:
      - VPC
      - RouteTableCIDRBlock
      - VPCRegion
    - Label:
        default: MongoDB Atlas ApiKey Configuration
      Parameters:
      - PublicKey
      - PrivateKey
      - OrgId
    - Label:
        default: MongoDB Atlas Configuration
      Parameters:
      - ProjectName
      - ClusterName
      - ClusterRegion
      - ClusterInstanceSize
      - DatabaseUserRoleDatabaseName
    ParameterLabels:
      VPC:
        default: VPC-id to peer with Atlas cluster
      RouteTableCIDRBlock:
        default: CIDR block for app to use in VPC Peering.
      PublicKey:
        default: MongoDB Atlas API PublicKey
      PrivateKey:
        default: MongoDB Atlas API PrivateKey
      OrgId:
        default: MongoDB Atlas API OrgId
      ProjectName:
        default: Name of new Atlas Project
      ClusterName:
        default: Name of new cluster
      ClusterRegion:
        default: The AWS Region for Atlas Cluster
      ClusterInstanceSize:
        default: MongoDB Atlas Instance Size
      DatabaseUserRoleDatabaseName:
        default: MongoDB Atlas Database User Role Database Name
      VPCRegion:
        default: The AWS Region for VPC
Parameters:
  PublicKey:
    Description: "Your MongoDB Cloud Public API Key"
    Type: String
    Default: "PublicKey"
  PrivateKey:
    Description: "Your MongoDB Cloud Private API Key"
    Type: String
    Default: "PrivateKey"
  OrgId:
    Description: "Your MongoDB Cloud Organization Id"
    Type: String
    Default: "OrgId"
  ProjectName:
    Description: "The name of the project."
    Type: String
    Default: "aws-quickstart-vpc"
  ClusterName:
    Description: Name of the cluster as it appears in Atlas. Once the cluster is created,
      its name cannot be changed.
    Type: String
    Default: "Cluster-1"
  ClusterInstanceSize:
    Default: "M10"
    Description: Atlas provides different cluster tiers, each with a default storage capacity and RAM size. The cluster you select is used for all the data-bearing hosts in your cluster tier. See https://docs.atlas.mongodb.com/reference/amazon-aws/#amazon-aws.
    Type: String
    AllowedValues:
    - "M10"
    - "M20"
    - "M30"
    - "M40"
    - "R40"
    - "M40_NVME"
    - "M50"
    - "R50"
    - "M50_NVME"
    - "M60"
    - "R60"
    - "M60_NVME"
    - "M80"
    - "R80"
    - "M80_NVME"
    - "M100"
    - "M140"
    - "M200"
    - "R200"
    - "M200_NVME"
    - "M300"
    - "R300"
    - "R400"
    - "M400_NVME"
    - "R700"
  ClusterRegion:
    Default: "US_EAST_1"
    Description: AWS Region where the Atlas database runs.
    Type: String
    AllowedValues:
      - "US_EAST_1"
      - "US_EAST_2"
      - "CA_CENTRAL_1"
      - "US_WEST_1"
      - "US_WEST_2"
      - "SA_EAST_1"
      - "AP_SOUTH_1"
      - "AP_EAST_2"
      - "AP_SOUTHEAST_1"
      - "AP_SOUTHEAST_2"
      - "AP_NORTHEAST_1"
      - "AP_NORTHEAST_2"
      - "EU_CENTRAL_1"
      - "EU_WEST_1"
      - "EU_NORTH_1"
      - "EU_WEST_2"
      - "EU_WEST_3"
      - "EU_SOUTH_1"
      - "ME_SOUTH_1"
      - "AF_SOUTH_1"
  ClusterMongoDBMajorVersion:
    Description: The version of MongoDB
    Type: String
    Default: "5.0"
    AllowedValues:
      - "4.0"
      - "4.2"
      - "4.4"
      - "5.0"
  DatabaseUserRoleDatabaseName:
    Description: Database User Role Database Name
    Type: String
    Default: "test"
  VPC:
    Type: AWS::EC2::VPC::Id
    Description: VPC-ID of your existing Virtual Private Cloud (VPC) which you wish to peer to your new MongoDB Atlas cluster. This is the VPC that your application uses usually.
  RouteTableCIDRBlock:
    Type: String
    Description: CIDR block to use for your VPC Peering.
  VPCRegion:
    Default: 'us-east-1'
    Description: AWS Region where the Quick Start S3 bucket (QSS3BucketName) is hosted. If you use your own bucket, you must specify this value.
    Type: String
Resources:
  AtlasIAMRole:
    DeletionPolicy: Retain
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
            Action: 'sts:AssumeRole'
  AtlasProject:
    DeletionPolicy: Retain
    Type: MongoDB::Atlas::Project
    DependsOn: AtlasIAMRole
    Properties:
      OrgId: !Ref "OrgId"
      ApiKeys:
        PublicKey:  !Ref PublicKey
        PrivateKey: !Ref PrivateKey
      Name: !Ref "ProjectName"
  AtlasProjectIPAccessList:
    DeletionPolicy: Retain
    Type: MongoDB::Atlas::ProjectIpAccessList
    DependsOn: AtlasProject
    Properties:
      ProjectId: !Ref AtlasProject
      ApiKeys:
        PublicKey:  !Ref PublicKey
        PrivateKey: !Ref PrivateKey
      AccessList:
        - IPAddress: "0.0.0.0/1"
          Comment: "Testing open all ips"
  AtlasNetworkPeering:
    DeletionPolicy: Retain
    Type: MongoDB::Atlas::NetworkPeering
    DependsOn: AtlasProject
    Properties:
      ProjectId: !Ref AtlasProject
      ApiKeys:
        PublicKey:  !Ref PublicKey
        PrivateKey: !Ref PrivateKey
      AccepterRegionName: !Ref "VPCRegion"
      AwsAccountId: !Sub "${AWS::AccountId}"
      RouteTableCIDRBlock: !Ref "RouteTableCIDRBlock"
      VpcId: !Ref "VPC"
  AtlasCluster:
    DeletionPolicy: Retain
    Type: MongoDB::Atlas::Cluster
    DependsOn: AtlasProject
    Properties:
      ApiKeys:
        PublicKey: !Ref PublicKey
        PrivateKey: !Ref PrivateKey
      ProjectId: !Ref AtlasProject
      Name: !Ref ClusterName
      MongoDBMajorVersion: !Ref "ClusterMongoDBMajorVersion"
      ClusterType: "REPLICASET"
      ReplicationSpecs:
        - NumShards: '1'
          AdvancedRegionConfigs:
            - AutoScaling:
                DiskGB:
                  Enabled: 'true'
                Compute:
                  Enabled: 'false'
                  ScaleDownEnabled: 'false'
              AnalyticsSpecs:
                EbsVolumeType: STANDARD
                InstanceSize: !Ref "ClusterInstanceSize"
                NodeCount: '3'
              ElectableSpecs:
                EbsVolumeType: STANDARD
                InstanceSize: !Ref "ClusterInstanceSize"
                NodeCount: '3'
              ReadOnlySpecs:
                EbsVolumeType: STANDARD
                InstanceSize: !Ref "ClusterInstanceSize"
                NodeCount: '3'
              Priority: '7'
              RegionName: !Ref ClusterRegion
  AtlasDatabaseUser:
    DeletionPolicy: Retain
    Type: MongoDB::Atlas::DatabaseUser
    DependsOn: 
      - AtlasCluster
      - AtlasProject
      - AtlasIAMRole
    Properties:
      ProjectId: !Ref AtlasProject
      ApiKeys:
        PublicKey:  !Ref PublicKey
        PrivateKey: !Ref PrivateKey
      Username: !GetAtt AtlasIAMRole.Arn
      DatabaseName: "$external"
      AWSIAMType: "ROLE"
      Roles:
        - RoleName: "readWrite"
          DatabaseName: !Ref "DatabaseUserRoleDatabaseName"
      Scopes:
        - Name: !Sub "${AWS::StackName}"
          Type: "CLUSTER"

Steps to Reproduce

This can be reproduced by following this guide using valid Atlas identifiers and the above template.

Expected Behavior

That the stack is created using existing Atlas resources.

Actual Behavior

The request fails like so (the required fields appear to differ depending on what resources are validated first):
image

I think the validation on the current model when handling a read event may be incorrect. Because the model doesn't exist within CloudFormation it can't possibly have the required fields and so reports an error. Or they may be some issue with unmarshalling the parameters that are provided for the model.

Additional Context

I am needing to import existing resources because I have had to transition from previously registered Atlas resource types used by the aws-quickstart templates to the newly activated ones - an issue similar to what was expressed here #77.

This is a test

CloudFormation Template

This is a test.

# Copy-paste your templates info here

Steps to Reproduce

Expected Behavior

Actual Behavior

Additional Context

References

IP not allowed in ProjectIpAccessList

I have tried to setup a IP access list in cloudformation to allow IPs from our office and other places by default. I have also tried to add the CIDR-blocks from out VPC peering into the same resource which works, but when I add the IP:s it doesn't work.

CloudFormation Template

"AtlasIpAccessList": {
   "Type": "MongoDB::Atlas::ProjectIpAccessList",
   "Properties": {
    "ProjectId": {
     "Ref": "AtlasProject"
    },
    "ApiKeys": {
     "PublicKey": {
      "Ref": "importedmongodbatlaspublickeyParameter"
     },
     "PrivateKey": {
      "Ref": "importedmongodbatlasprivatekeyParameter"
     }
    },
    "AccessList": [
     {
      "Comment": "Office IP 1 ",
      "IPAddress": "xxx.xxx.xxx.xxx/23"
     },
     {
      "Comment": "Office IP 2 ",
      "IPAddress": "yyy.yyy.yyy.yyy/23"
     },
     {
      "CIDRBlock": {
       "Fn::ImportValue": "hfs2-dev-vpc-CidrBlock"
      },
      "Comment": "Fleet VPC "
     },
     {
      "CIDRBlock": {
       "Fn::ImportValue": "databricks-vpc-CidrBlock"
      },
      "Comment": "Fleet Databricks VPC"
     }
    ]
   },
   "DependsOn": [
    "DatabricksNetworkPeering",
    "FleetNetworkPeering"
   ]
  }

Steps to Reproduce

Try adding an IP into the ProjectIpAccessList resource.

Expected Behavior

The IP should be added alongside the cidr blocks into the projects whitelisting in Atlas.

Actual Behavior

Getting the following error:

21:44:52 | UPDATE_FAILED        | MongoDB::Atlas::ProjectIpAccessList | dev/AtlasIpAccessList (AtlasIpAccessList) Resource handler returned message: "Error getting resource : DELETE https://cloud.mongodb.com/api/atlas/v1.0/groups/63d454d5bcf2a040f70f7bae/accessList/xxx.xxx.xxx.xxx%2F23: 404 (request "ATLAS_NETWORK_PERMISSION_ENTRY_NOT_FOUND") IP Address xxx.xxx.xxx.xxx/23 not on Atlas access list for group 63d454d5bcf2a040f70f7bae." (RequestToken: 1e5219b8-24fe-8c0b-58d1-b2fff9fe1788, HandlerErrorCode: NotFound)
 ❌  dev (hfs2-dev-mongo-db-atlas) failed: Error: The stack named hfs2-dev-mongo-db-atlas failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Error getting resource : DELETE https://cloud.mongodb.com/api/atlas/v1.0/groups/63d454d5bcf2a040f70f7bae/accessList/xxx.xxx.xxx.xxx%2F23: 404 (request "ATLAS_NETWORK_PERMISSION_ENTRY_NOT_FOUND") IP Address xxx.xxx.xxx.xxx/23 not on Atlas access list for group 63d454d5bcf2a040f70f7bae." (RequestToken: 1e5219b8-24fe-8c0b-58d1-b2fff9fe1788, HandlerErrorCode: NotFound)
    at FullCloudFormationDeployment.monitorDeployment (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\api\deploy-stack.ts:505:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at deployStack2 (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\cdk-toolkit.ts:265:24)
    at C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\deploy.ts:39:11
    at run (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\p-queue\dist\index.js:163:29)
 ❌ Deployment failed: Error: Stack Deployments Failed: Error: The stack named hfs2-dev-mongo-db-atlas failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Error getting resource : DELETE https://cloud.mongodb.com/api/atlas/v1.0/groups/63d454d5bcf2a040f70f7bae/accessList/xxx.xxx.xxx.xxx%2F23: 404 (request "ATLAS_NETWORK_PERMISSION_ENTRY_NOT_FOUND") IP Address xxx.xxx.xxx.xxx/23 not on Atlas access list for group 63d454d5bcf2a040f70f7bae." (RequestToken: 1e5219b8-24fe-8c0b-58d1-b2fff9fe1788, HandlerErrorCode: NotFound)      
    at deployStacks (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\deploy.ts:61:11)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at CdkToolkit.deploy (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\cdk-toolkit.ts:339:7)
    at exec4 (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\cli.ts:384:12)
Stack Deployments Failed: Error: The stack named hfs2-dev-mongo-db-atlas failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Error getting resource : DELETE https://cloud.mongodb.com/api/atlas/v1.0/groups/63d454d5bcf2a040f70f7bae/accessList/xxx.xxx.xxx.xxx%2F23: 404 (request "ATLAS_NETWORK_PERMISSION_ENTRY_NOT_FOUND") IP Address xxx.xxx.xxx.xxx/23 not on Atlas access list for group 63d454d5bcf2a040f70f7bae." (RequestToken: 1e5219b8-24fe-8c0b-58d1-b2fff9fe1788, HandlerErrorCode: NotFound)

Additional Context

I am using CDK but manually creating the objects as CfnResource classes. My examples are from the cdk synth output since that is what will be uploaded to AWS which is the relevant part. I have tried to remove the cdk specific stuff from the template to keep it clean for the issue but there are some cdk references in the error log, but that should not be relevant for this issue as long as the output from cdk synth is correct.

Creating change set fails with internal error

CloudFormation Template

AWSTemplateFormatVersion: 2010-09-09
Conditions:
  PROD:
    Fn::Equals:
    - Ref: Environment
    - PROD
Description: MongoDB Atlas
Metadata:
  StackName: MongoDBAtlas
Outputs:
  AtlasAPISecretName:
    Description: Cluster Id is also a name for secret with API keys
    Export:
      Name:
        Fn::Sub: ${AWS::StackName}-AtlasAPISecretName
    Value:
      Ref: Cluster
  ConnectionStringSrv:
    Description: Cluster connection strings
    Export:
      Name:
        Fn::Sub: ${AWS::StackName}-ConnectionStringSrv
    Value:
      Fn::GetAtt:
      - Cluster
      - ConnectionStrings.StandardSrv
  Role:
    Description: ARN of AWS IAM Role database cluster access
    Export:
      Name:
        Fn::Sub: ${AWS::StackName}-Role
    Value:
      Fn::GetAtt:
      - IAMRole
      - Arn
  SecretArn:
    Description: ARN of MongoDB Atlas root user secret
    Export:
      Name:
        Fn::Sub: ${AWS::StackName}-SecretArn
    Value:
      Ref: Secret
Parameters:
  BuildDir:
    Description: Folder name with application build
    Type: String
  DataDogApiKey:
    Description: DataDog api key for lambda logs forwarding
    NoEcho: true
    Type: String
  Environment:
    Description: Stack creates lambdas with prefixes taken from environment variable
      value
    Type: String
  NetworkEnv:
    Description: Prefix used to specify the network name
    Type: String
  OrganizationId:
    Description: MongoDB Atlas Organization Id
    Type: String
  PrivateKey:
    Description: MongoDB Atlas Private API Key
    NoEcho: true
    Type: String
  PublicKey:
    Description: MongoDB Atlas Public API Key
    NoEcho: true
    Type: String
  Version:
    Description: Current version
    Type: String
Resources:
  AtlasManager:
    Properties:
      PrivateKey:
        Ref: PrivateKey
      ProjectId:
        Ref: Project
      PublicKey:
        Ref: PublicKey
      ServiceToken:
        Fn::GetAtt:
        - AtlasManagerLambda
        - Arn
      VpcPeeringId:
        Ref: NetworkPeering
    Type: Custom::MongoDBAtlasManager
    Version: "0.1"
  AtlasManagerLambda:
    DependsOn:
    - AtlasManagerLambdaRole
    Properties:
      Code:
        S3Bucket:
          Fn::Transform:
            Name: String
            Parameters:
              InputString:
                Fn::Sub: ${Environment}.sc.assets
              Operation: BuildBucket
        S3Key:
          Fn::Sub: mongodbatlas/${BuildDir}/lambdas/atlas_manager/bootstrap.zip
      Description: AWS Lambda function
      Environment:
        Variables:
          DD_API_KEY:
            Ref: DataDogApiKey
          DD_ENV:
            Ref: Environment
          DD_VERSION:
            Ref: Version
          Environment:
            Ref: Environment
      FunctionName:
        Fn::Sub: ${Environment}-AtlasManager-Lambda
      Handler: bootstrap
      Layers:
      - Fn::If:
        - PROD
        - arn:aws:lambda:us-east-1:464622532012:layer:Datadog-Extension:35
        - Ref: AWS::NoValue
      MemorySize: 256
      Role:
        Fn::GetAtt:
        - AtlasManagerLambdaRole
        - Arn
      Runtime: provided.al2
      Timeout: 900
      TracingConfig:
        Mode: Active
      VpcConfig:
        SecurityGroupIds:
        - Fn::ImportValue:
            Fn::Sub: ${NetworkEnv}-Network-PrivateNatSecurityGroupId
        SubnetIds:
          Fn::Split:
          - ','
          - Fn::ImportValue:
              Fn::Sub: ${NetworkEnv}-Network-PrivateNatSubnetIds
    Type: AWS::Lambda::Function
  AtlasManagerLambdaRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          Action: sts:AssumeRole
          Effect: Allow
          Principal:
            Service: lambda.amazonaws.com
        Version: 2012-10-17
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess
      - arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
      Path: /
      Policies:
      - PolicyDocument:
          Statement:
          - Action:
            - ec2:DescribeVpcPeeringConnections
            Effect: Allow
            Resource: '*'
          - Action:
            - ec2:AcceptVpcPeeringConnection
            Effect: Allow
            Resource:
            - Fn::Sub: arn:aws:ec2:${AWS::Region}:${AWS::AccountId}:vpc/*
            - Fn::Sub: arn:aws:ec2:${AWS::Region}:${AWS::AccountId}:vpc-peering-connection/*
          Version: 2012-10-17
        PolicyName: HandleVPCPeeringConnection
      RoleName:
        Fn::Sub: ${Environment}-AtlasManager-LambdaRole
      Tags:
      - Key: env
        Value:
          Fn::Sub: ${Environment}
      - Key: service
        Value: mongodbatlas
      - Key: version
        Value:
          Fn::Sub: ${Version}
    Type: AWS::IAM::Role
  Cluster:
    Properties:
      ApiKeys:
        PrivateKey:
          Ref: PrivateKey
        PublicKey:
          Ref: PublicKey
      BackupEnabled: true
      ClusterType: REPLICASET
      DiskSizeGB: 50
      MongoDBMajorVersion: "6.0"
      Name:
        Fn::Sub: ${Environment}-SmartMerchant-Cluster
      PitEnabled: false
      ProjectId:
        Ref: Project
      ReplicationSpecs:
      - AdvancedRegionConfigs:
        - AutoScaling:
            DiskGB:
              Enabled: true
          ElectableSpecs:
            EbsVolumeType: PROVISIONED
            InstanceSize: M10
            NodeCount: 3
          Priority: 7
          RegionName: US_EAST_1
        NumShards: 0
      TerminationProtectionEnabled: true
    Type: MongoDB::Atlas::Cluster
    Version: "0.1"
  IAMRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          Action: sts:AssumeRole
          Effect: Allow
          Principal:
            AWS:
              Fn::Sub: arn:aws:iam::${AWS::AccountId}:root
        Version: 2012-10-17
      RoleName:
        Fn::Sub: ${Environment}-MongoDBAtlas-Role
    Type: AWS::IAM::Role
  IAMRoleDatabaseUser:
    Properties:
      AWSIAMType: ROLE
      ApiKeys:
        PrivateKey:
          Ref: PrivateKey
        PublicKey:
          Ref: PublicKey
      DatabaseName: $external
      ProjectId:
        Ref: Project
      Roles:
      - DatabaseName: admin
        RoleName: readWriteAnyDatabase
      Username:
        Fn::GetAtt:
        - IAMRole
        - Arn
    Type: MongoDB::Atlas::DatabaseUser
    Version: "0.1"
  NetworkPeering:
    Properties:
      AccepterRegionName:
        Fn::Sub: ${AWS::Region}
      ApiKeys:
        PrivateKey:
          Ref: PrivateKey
        PublicKey:
          Ref: PublicKey
      AwsAccountId:
        Fn::Sub: ${AWS::AccountId}
      ProjectId:
        Ref: Project
      RouteTableCIDRBlock:
        Fn::ImportValue:
          Fn::Sub: ${NetworkEnv}-Network-PrivateVPCCidrBlock
      VpcId:
        Fn::ImportValue:
          Fn::Sub: ${NetworkEnv}-Network-PrivateVPCId
    Type: MongoDB::Atlas::NetworkPeering
    Version: "0.1"
  PasswordDatabaseUser:
    Properties:
      AWSIAMType: NONE
      ApiKeys:
        PrivateKey:
          Ref: PrivateKey
        PublicKey:
          Ref: PublicKey
      DatabaseName: admin
      Password:
        Fn::Join:
        - ""
        - - '{{resolve:secretsmanager:'
          - Ref: Secret
          - :SecretString:password}}
      ProjectId:
        Ref: Project
      Roles:
      - DatabaseName: admin
        RoleName: readWriteAnyDatabase
      Username:
        Fn::Join:
        - ""
        - - '{{resolve:secretsmanager:'
          - Ref: Secret
          - :SecretString:username}}
    Type: MongoDB::Atlas::DatabaseUser
    Version: "0.1"
  PrivateNatVPCRoute:
    DependsOn:
    - AtlasManager
    Properties:
      DestinationCidrBlock:
        Fn::GetAtt:
        - AtlasManager
        - AtlasCidr
      RouteTableId:
        Fn::ImportValue:
          Fn::Sub: ${NetworkEnv}-Network-PrivateNatVPCRouteTableId
      VpcPeeringConnectionId:
        Fn::GetAtt:
        - AtlasManager
        - VpcPeeringConnectionID
    Type: AWS::EC2::Route
  PrivateVPCRoute:
    DependsOn:
    - AtlasManager
    Properties:
      DestinationCidrBlock:
        Fn::GetAtt:
        - AtlasManager
        - AtlasCidr
      RouteTableId:
        Fn::ImportValue:
          Fn::Sub: ${NetworkEnv}-Network-PrivateVPCRouteTableId
      VpcPeeringConnectionId:
        Fn::GetAtt:
        - AtlasManager
        - VpcPeeringConnectionID
    Type: AWS::EC2::Route
  Project:
    Properties:
      ApiKeys:
        PrivateKey:
          Ref: PrivateKey
        PublicKey:
          Ref: PublicKey
      Name:
        Fn::Sub: ${Environment}-SmartMerchant
      OrgId:
        Ref: OrganizationId
    Type: MongoDB::Atlas::Project
    Version: "0.1"
  ProjectIPAccessList:
    DependsOn:
    - NetworkPeering
    - AtlasManager
    Properties:
      AccessList:
      - AwsSecurityGroup:
          Fn::ImportValue:
            Fn::Sub: ${NetworkEnv}-Network-PrivateSecurityGroupId
        Comment: Access for private security group.
        ProjectId:
          Ref: Project
      - AwsSecurityGroup:
          Fn::ImportValue:
            Fn::Sub: ${NetworkEnv}-Network-PrivateNatSecurityGroupId
        Comment: Access for private NAT security group.
        ProjectId:
          Ref: Project
      ApiKeys:
        PrivateKey:
          Ref: PrivateKey
        PublicKey:
          Ref: PublicKey
      ProjectId:
        Ref: Project
    Type: MongoDB::Atlas::ProjectIpAccessList 
    Version: "0.1"
  Secret:
    Properties:
      Description: Secret for MongoDB Atlas cluster
      GenerateSecretString:
        ExcludePunctuation: true
        GenerateStringKey: password
        PasswordLength: 16
        SecretStringTemplate:
          Fn::Join:
          - ""
          - - '{"username":"root",'
            - '"connectionString":"'
            - Fn::GetAtt:
              - Cluster
              - ConnectionStrings.StandardSrv
            - '"}'
      Name:
        Fn::Sub: ${Environment}/MongoDBAtlas-Secret
    Type: AWS::SecretsManager::Secret

Steps to Reproduce

Create change set using either CLI or GUI.
Command for CLI:

aws cloudformation create-change-set --stack-name ${ENV}-MongoDBAtlas \
--template-url url --parameters \
ParameterKey=Environment,ParameterValue=${ENV} ParameterKey=NetworkEnv,ParameterValue=${NETWORK_ENV} \
ParameterKey=Version,ParameterValue=${VERSION} ParameterKey=BuildDir,ParameterValue=${BUILD_DIR} \
ParameterKey=PublicKey,UsePreviousValue=true \
ParameterKey=PrivateKey,UsePreviousValue=true \
ParameterKey=OrganizationId,UsePreviousValue=true \
ParameterKey=DataDogApiKey,UsePreviousValue=true \
--no-use-previous-template \
--change-set-name MongoDBAtlas-ChangeSet${BITBUCKET_BUILD_NUMBER} \
--capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_NAMED_IAM

Expected Behavior

Change set creation should ends with success or describe what went wrong.

Actual Behavior

Returned response is
There was an error creating this change set Internal Failure
Execution status: UNAVAILABLE
Status reason: Internal Failure

Additional Context

References

Hiccups when waiting for MongoDB::Atlas::Cluster to register

Created Project, added DatabaseUser through CloudFormation and was trying to add Cluster resource.

In first attempt, stack was about 5 minutes in CREATE_IN_PROGRESS state and then it returned following error:

Unable to complete request: error fetching cluster info (cfn-built-cluster-azec): Get "https://cloud.mongodb.com/api/atlas/v1.0/groups/5ebd98e75da9fc3cefdb8a0a/clusters/cfn-built-cluster-azec": unexpected EOF

Here is more info about my env.

> pip show cloudformation-cli

Name: cloudformation-cli
Version: 0.1.3
Summary: UNKNOWN
Home-page: https://github.com/aws-cloudformation/aws-cloudformation-rpdk/
Author: Amazon Web Services
Author-email: [email protected]
License: Apache License 2.0
Location: /Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages
Requires: jsonschema, Werkzeug, colorama, requests, hypothesis, pytest, PyYAML, boto3, Jinja2
Required-by: cloudformation-cli-go-plugin
pip show cloudformation-cli-go-plugin

Name: cloudformation-cli-go-plugin
Version: 0.1.2
Summary: UNKNOWN
Home-page: https://github.com/aws-cloudformation/cloudformation-cli-go-plugin/
Author: Amazon Web Services
Author-email: [email protected]
License: Apache License 2.0
Location: /Users/azec/.pyenv/versions/3.7.4/lib/python3.7/site-packages
Requires: cloudformation-cli
Required-by: 

CloudFormation template used

AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Cloudformation template to create Atlas resources in MongoDB Cloud Manager

Parameters:
  accountAlias:
    Type: "AWS::SSM::Parameter::Value<String>"
    Default: '/regionSettings/accountAlias'

Resources:
  #Atlas Project
  MongoDBAtlasProject:
    Type: 'MongoDB::Atlas::Project'
    Properties:
      Name: 'CBT Atlas QA (azec)'
      OrgId: '599eecea9f78f769464d1708'  #TODO: Put in SSM parameter
      ApiKeys:  # NOTE: These have to be configured API Keys on ORG level in Atlas
        PublicKey: '{{resolve:secretsmanager:/administration/atlas/shared/org-api-key:SecretString:public-key}}'
        PrivateKey: '{{resolve:secretsmanager:/administration/atlas/shared/org-api-key:SecretString:private-key}}'
  
  # Atlas database user
  MongoDBAtlasDBUser:
    Type: 'MongoDB::Atlas::DatabaseUser'
    Properties:
      Username: 'maskedUsername'
      Password: 'maskedPass'
      ProjectId: !Ref MongoDBAtlasProject
      DatabaseName: 'admin'
      Roles: 
        - 
          RoleName: 'atlasAdmin'
          DatabaseName: 'admin'
      ApiKeys:  # NOTE: These get propagated to Project level from ORG level in Atlas, but we store them in separate secret (same values) in case we ever need to decouple
        PublicKey: '{{resolve:secretsmanager:/administration/atlas/shared/project-api-key:SecretString:public-key}}'
        PrivateKey: '{{resolve:secretsmanager:/administration/atlas/shared/project-api-key:SecretString:private-key}}'

  # Atlas cluster
  MongoDBAtlasCluster:
    Type: 'MongoDB::Atlas::Cluster'
    Properties:
      ProjectId: !Ref MongoDBAtlasProject
      Name: 'cfn-built-cluster-azec'
      NumShards: 1
      ProviderName: 'AWS'
      ReplicationFactor: 3
      ProviderBackupEnabled: false
      AutoScaling:
        DiskGBEnabled: false
      MongoDBVersion: '4.2'
      ProviderSettings:
        VolumeType: 'STANDARD'
        EncryptEBSVolume: false
        InstanceSizeName: 'M10'
        RegionName: "US_EAST_1"
      ApiKeys:  # NOTE: These get propagated to Project level from ORG level in Atlas, but we store them in separate secret (same values) in case we ever need to decouple
        PublicKey: '{{resolve:secretsmanager:/administration/atlas/shared/project-api-key:SecretString:public-key}}'
        PrivateKey: '{{resolve:secretsmanager:/administration/atlas/shared/project-api-key:SecretString:private-key}}'

During this error, Cluster went int auto-shutdown mode and eventually disappeared in Atlas.

Screen Shot 2020-05-14 at 1 52 25 PM

After my 2nd attempt to stack cluster , without any change in the template, it went fine.
However, I believe there is opportunity in how waiting for cluster to become available is implemented in MongoDB::Atlas::Cluster handler.

This is a test

CloudFormation Template

# Copy-paste your templates info here

Steps to Reproduce

Expected Behavior

Actual Behavior

Additional Context

References

Reference vpc peering connectionId not working

I am trying to attach a route table to my VPC peering connection in the same template as where I setup the VPC peering and it doesn't allow me to read the property from the NetworkPeering object.

CloudFormation Template

These are the relevant parts from my CloudFormation template.

"DatabricksNetworkPeering": {
   "Type": "MongoDB::Atlas::NetworkPeering",
   "Properties": {
    "ApiKeys": {
     "PublicKey": {
      "Ref": "importedmongodbatlaspublickeyParameter"
     },
     "PrivateKey": {
      "Ref": "importedmongodbatlasprivatekeyParameter"
     }
    },
    "ProjectId": {
     "Ref": "AtlasProject"
    },
    "ContainerId": {
     "Ref": "AtlasNetworkContainer"
    },
    "AccepterRegionName": {
     "Ref": "AWS::Region"
    },
    "AwsAccountId": {
     "Ref": "AWS::AccountId"
    },
    "RouteTableCIDRBlock": {
     "Fn::ImportValue": "databricks-vpc-CidrBlock"
    },
    "VpcId": {
     "Fn::ImportValue": "databricks-vpc-VpcId"
    }
   }
  },
"RouteFromPrivateSubnetOfDatabricksToMongoDBOne": {
   "Type": "AWS::EC2::Route",
   "Properties": {
    "RouteTableId": {
     "Fn::ImportValue": "databricks-network-PrivateSubnetOneRouteTableId"
    },
    "DestinationCidrBlock": {
      "Ref": "monogdbCidrBlock"
     },
    "VpcPeeringConnectionId": {
     "Fn::GetAtt": [
      "DatabricksNetworkPeering",
      "ConnectionId"
     ]
    }
   },
   "DependsOn": [
    "DatabricksNetworkPeering"
   ]
  }

Steps to Reproduce

Create a VPC peering and a EC2 Route in the same template and try to get the connectionId from the network peering.

Expected Behavior

An id that starts with "pcx-" should be returned from the GetAtt ConnectionId

Actual Behavior

Getting this issue:

15:09:09 | CREATE_FAILED        | AWS::EC2::Route                     | dev/RouteFromPrivateSubnetOfDatabricksToMongoDBOne (RouteFromPrivateSubnetOfDatabricksToMongoDBOne) Requested attribute ConnectionId must be a readonly property in schema for MongoDB::Atlas::NetworkPeering       

 ❌  dev (hfs2-dev-mongo-db-atlas) failed: Error: The stack named hfs2-dev-mongo-db-atlas failed to deploy: UPDATE_ROLLBACK_COMPLETE: Requested attribute ConnectionId must be a readonly property in schema for MongoDB::Atlas::NetworkPeering, Requested attribute ConnectionId must be a readonly property in schema for MongoDB::Atlas::NetworkPeering    at FullCloudFormationDeployment.monitorDeployment (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\api\deploy-stack.ts:505:13)    
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at deployStack2 (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\cdk-toolkit.ts:265:24)
    at C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\deploy.ts:39:11
    at run (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\p-queue\dist\index.js:163:29)

 ❌ Deployment failed: Error: Stack Deployments Failed: Error: The stack named hfs2-dev-mongo-db-atlas failed to deploy: UPDATE_ROLLBACK_COMPLETE: Requested attribute ConnectionId must be a readonly property in schema for MongoDB::Atlas::NetworkPeering, Requested attribute ConnectionId must be a readonly property in schema for MongoDB::Atlas::NetworkPeering    at deployStacks (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\deploy.ts:61:11)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at CdkToolkit.deploy (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\cdk-toolkit.ts:339:7)
    at exec4 (C:\Users\matsp\AppData\Roaming\nvm\v16.17.1\node_modules\aws-cdk\lib\cli.ts:384:12)

Stack Deployments Failed: Error: The stack named hfs2-dev-mongo-db-atlas failed to deploy: UPDATE_ROLLBACK_COMPLETE: Requested attribute ConnectionId must be a readonly property in schema for MongoDB::Atlas::NetworkPeering, Requested attribute ConnectionId must be a readonly property in schema for MongoDB::Atlas::NetworkPeering

Unable to retrieve ConnectionStrings attribute for MongoDB::Atlas::Cluster

CloudFormation Template

Resources:
  Cluster:
    Properties:
      ApiKeys:
        PrivateKey:
          Ref: PrivateKey
        PublicKey:
          Ref: PublicKey
      BackupEnabled: false
      ClusterType: REPLICASET
      MongoDBMajorVersion: "6.0"
      Name:
        Fn::Sub: ${Environment}-SmartMerchant-Cluster
      PitEnabled: false
      ProjectId:
        Ref: Project
      ReplicationSpecs:
      - AdvancedRegionConfigs:
        - ElectableSpecs:
            EbsVolumeType: PROVISIONED
            InstanceSize: M10
            NodeCount: 3
          Priority: 7
          RegionName: US_EAST_1
        NumShards: 0
      TerminationProtectionEnabled: false
    Type: MongoDB::Atlas::Cluster
    Version: "0.1"

Outputs:
  MongoDBAtlasConnectionStrings:
    Description: Cluster connection strings
    Export:
      Name:
        Fn::Sub: ${AWS::StackName}-ConnectionStrings
    Value:
      Fn::GetAtt:
      - Cluster
      - ConnectionStrings

Steps to Reproduce

I am using the official public extensions for MongoDB Atlas in CloudFormation and I have successfully deployed all the resources, however when I am trying to output some properties of the cluster, deployment fails.

Expected Behavior

The output parameters like ConnectionStrings or ClusterState should be available.

Actual Behavior

I am receiving an error Unable to retrieve ConnectionStrings attribute for MongoDB::Atlas::Cluster, with error message The next fields are required ApiKeys.PublicKey ApiKeys.PrivateKey ProjectId Name

Additional Context

Secret in Secrets Manager which is created along with the Cluster is present.

empty readOnlyProperties in database user schema causes error

Got error trying to build the database user resource.

[2020-06-01T19:07:07Z] DEBUG    - Writing Makefile: /home/jason/work/mongodbatlas-cloudformation-resources/database-user/Make
file
[2020-06-01T19:07:07Z] DEBUG    - Overwriting '/home/jason/work/mongodbatlas-cloudformation-resources/database-user/Makefile'
[2020-06-01T19:07:07Z] DEBUG    - Removing generated docs: /home/jason/work/mongodbatlas-cloudformation-resources/database-us
er/docs
[2020-06-01T19:07:07Z] DEBUG    - Writing generated docs
[2020-06-01T19:07:07Z] DEBUG    - Writing docs labels.md: /home/jason/work/mongodbatlas-cloudformation-resources/database-use
r/docs/labels.md
[2020-06-01T19:07:07Z] DEBUG    - Writing docs roles.md: /home/jason/work/mongodbatlas-cloudformation-resources/database-user
/docs/roles.md
[2020-06-01T19:07:07Z] DEBUG    - Writing docs apikeys.md: /home/jason/work/mongodbatlas-cloudformation-resources/database-us
er/docs/apikeys.md
[2020-06-01T19:07:07Z] DEBUG    - Finished documenting nested properties
[2020-06-01T19:07:07Z] WARNING  - prop=
[2020-06-01T19:07:07Z] WARNING  - path=()
[2020-06-01T19:07:07Z] DEBUG    - Unhandled exception
Traceback (most recent call last):
  File "/home/jason/.local/lib/python3.7/site-packages/rpdk/core/cli.py", line 98, in main
    args.command(args)
  File "/home/jason/.local/lib/python3.7/site-packages/rpdk/core/generate.py", line 16, in generate
    project.generate_docs()
  File "/home/jason/.local/lib/python3.7/site-packages/rpdk/core/project.py", line 373, in generate_docs
    getatt = self._get_docs_gettable_atts(docs_schema)
  File "/home/jason/.local/lib/python3.7/site-packages/rpdk/core/project.py", line 415, in _get_docs_gettable_atts
    for prop in docs_schema.get("readOnlyProperties", [])
  File "/home/jason/.local/lib/python3.7/site-packages/rpdk/core/project.py", line 415, in <listcomp>
    for prop in docs_schema.get("readOnlyProperties", [])
  File "/home/jason/.local/lib/python3.7/site-packages/rpdk/core/project.py", line 404, in _get_property_description
    name = path[-1]
IndexError: tuple index out of range

I was able to fix this by adding

   "readOnlyProperties": [
        "/properties/DatabaseName",
        "/properties/ProjectId",
        "/properties/Username"
    ],

to the database user schema seems to fix.
See

Thanks-

Potencial Security Issue: the api keys are being returned in the GET operation

Both the private and public key are being returned in get operation, leading to a potencial key leaking.

The keys can be desplayed in the parameter tab.

image

This issue can be mitigated, always suggesting the user to use Secrets to store the keys. But the keys can be access if the user decide to Output them.

the best solution, would be to access the secrets directly in the resource code, and not return them or use them as an input model.

for this AWS suggest to use typeConfiguration: https://docs.aws.amazon.com/cloudformation-cli/latest/userguide/resource-type-model.html

this way we are forcing the user to always provide the keys as secrets, and not being able to output them in any way.

CFN resources versioning

Hello

In the last week you've made changes to the atlas cfn resources that change the way the resources are to be created.
These changes are deployed directly to the public third party extensions and this breaks any previously developed stacks.

Are you able to use versioning here so that your breaking changes don't directly impact existing stacks?

MongoDB::Atlas::PrivateEndpoint schema incorrectly structure to only allow 1-1 connection between vpc and subnet

The MongoDB::Atlas::PrivateEndpoint custom resource schema seems to be incorrectly structured to only allow one subnetId per VPC when setting up the private endpoint.

In our use case we have an ECS Cluster with tasks split across three subnets. I want to create a Private Endpoint that links to the three subnets. However the current schema structure is 1-1 and therefore we end up setting up 3 private endpoints, one for each subnet.

This should replicate the functionality that is available in the Atlas console. i.e. attaching multiple subnets to a VPC when creating the Private Endpoint as seen in the screenshot below:
Screenshot 2023-01-18 at 4 01 40 PM

Our current solution looks like this:

 AtlasPrivateEndpoint:
    DependsOn: AtlasProject
    Type: MongoDB::Atlas::PrivateEndpoint
    Properties:
      GroupId: !Ref "AtlasProject"
      ApiKeys:
        PublicKey:  !Ref "PublicKey"
        PrivateKey: !Ref "PrivateKey"
      Region: !Ref "ClusterRegion"
      PrivateEndpoints:
      - VpcId: 
          Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEVPC']]
        SubnetId: 
          Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEPublicSubnet1']]
      - VpcId: 
          Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEVPC']]
        SubnetId: 
          Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEPublicSubnet2']]
      - VpcId: 
          Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEVPC']]
        SubnetId: 
          Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEPublicSubnet3']]

However, as mentioned this sets up 3 private endpoints because the schema is defined as such https://github.com/mongodb/mongodbatlas-cloudformation-resources/blob/master/cfn-resources/private-endpoint/docs/privateendpoint.md

Expected Behaviour:
We should be able to attach multiple subnetIds to a singular VPCId from cloudformation, i would expect the use case above would then look something similar to this:

 AtlasPrivateEndpoint:
    DependsOn: AtlasProject
    Type: MongoDB::Atlas::PrivateEndpoint
    Properties:
      GroupId: !Ref "AtlasProject"
      ApiKeys:
        PublicKey:  !Ref "PublicKey"
        PrivateKey: !Ref "PrivateKey"
      Region: !Ref "ClusterRegion"
      PrivateEndpoints:
      - VpcId: 
          Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEVPC']]
          -SubnetIds: 
            Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEPublicSubnet1']]
            Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEPublicSubnet2']]
            Fn::ImportValue: !Join [':', [!Ref 'NetworkStackName', 'BEPublicSubnet3']]
``

Define Installation Steps for Dependencies

the build step for each resource example required cfn command, and there's an node package for it available, but that package doesn't support cnf generate or cnf init which are declared into Makefile for each resource kind.
It would be great to have it documented about how to install this cfn command,
At the time of writing this I am still not sure where to get it.

MongoDB::Atlas::Cluster doesn't expose all ProviderSettings available in Atlas API

After creating resources:

  • project
  • db user
    I was going to stack Cluster via CloudFormation. Kept failing on specification of cluster tier:
  • per Atlas docs of enumerations for available instance sizes - they don’t allow M0 Sandbox through MongoDB Atlas API at all (these are FREE and allowed only through UI)

Then started reading https://docs.atlas.mongodb.com/reference/api/clusters-create-one/
and found this really interesting:

providerSettings.instanceSizeName

M2 and M5 clusters are multi-tenant deployments. 
You must set providerSettings.providerName to TENANT and specify the cloud service provider in providerSettings.backingProviderName.

Looking at current spec of MongoDB::Atlas::Cluster , CloudFormation property ProviderSettings DOES NOT allow specifying ProviderName, only BackingProviderName property which is not enough to provision M2 and M5 tier clusters.

This creates constraint that minimum possible tier that can be provisioned through CloudFormation (and AWS provider - duuh ) is M10 Cluster.

I am curious if this is "by design" and intentional business decision vs. implementation opportunity? Answer to that question might determine if this issue should be treated as feature-request or canceled.

Additionally , I can see that example for cluster is using also M10.
However, the same example is using ProviderName as top-level property on resource here, although it is nowhere specified in JSON Schema for MongoDB::Atlas::Cluster and interestingly I did have it in my template (by pure copy-paste) and it didn't make resource fail :)

SUMMARY:

  1. There may be opportunity to expand ProviderSettings
  2. There is opportunity to clean up examples for MongoDB::Atlas::Cluster

Add support for Team and User

Please, add support for MongoDB::Atlas::Team and MongoDB::Atlas::User so that user provisioning and team assignment can be automated. This is also needed if you use SAML.

Register custom resources into AWS

I have a Azure DevOps Pipeline setup to run the "cfn-submit-helper.sh" script to setup the custom resources needed in the aws region to be able to use cloudformation to setup the MongoDB environment.
I managed to install all required libraries and get the script to run but ran into some issues.

Steps to Reproduce

Run the cfn-submit-helper.sh

Actual Behavior

This is the log I get from the step in Azure DevOps which runs this script:

[2022-11-11T14:52:26Z] WARNING  - Resource schema is valid.
[2022-11-11T14:52:26Z] INFO     - Validating your resource schema...
[2022-11-11T14:52:26Z] DEBUG    - Writing Execution Role CloudFormation template: /home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/resource-role.yaml
[2022-11-11T14:52:26Z] DEBUG    - Overwriting '/home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/resource-role.yaml'
[2022-11-11T14:52:26Z] DEBUG    - Generate started
[2022-11-11T14:52:26Z] DEBUG    - Writing Types
[2022-11-11T14:52:26Z] DEBUG    - Overwriting '/home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/cmd/resource/model.go'
[2022-11-11T14:52:26Z] DEBUG    - Writing project: /home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/cmd/main.go
[2022-11-11T14:52:26Z] DEBUG    - Overwriting '/home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/cmd/main.go'
[2022-11-11T14:52:26Z] DEBUG    - Writing makebuild: /home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/makebuild
[2022-11-11T14:52:26Z] DEBUG    - Overwriting '/home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/makebuild'
[2022-11-11T14:52:26Z] DEBUG    - Removing generated docs: /home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/docs
[2022-11-11T14:52:26Z] DEBUG    - Writing generated docs
[2022-11-11T14:52:26Z] DEBUG    - Writing docs accesslistdefinition.md: /home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/docs/accesslistdefinition.md
[2022-11-11T14:52:26Z] DEBUG    - Writing docs apikeydefinition.md: /home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/docs/apikeydefinition.md
[2022-11-11T14:52:26Z] DEBUG    - Finished documenting nested properties
[2022-11-11T14:52:26Z] DEBUG    - Writing docs README: /home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources/project-ip-access-list/docs/README.md
[2022-11-11T14:52:27Z] WARNING  - Generated files for MongoDB::Atlas::ProjectIpAccessList
[2022-11-11T14:52:27Z] DEBUG    - Finished generate
+ cd -
+ for resource in ${resources}
+ [[ false == \t\r\u\e ]]
+ echo 'Working on resource:network-peering'
/home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources
Working on resource:network-peering
++ pwd
+ cwd=/home/vsts/work/1/s/mongodbatlas-cloudformation-resources/cfn-resources
+ cd network-peering
+ echo 'resource: network-peering'
+ [[ debug == \d\e\b\u\g ]]
+ make debug
resource: network-peering
#make -f makebuild  # this runs build steps required by the cfn cli
cfn generate
Explicitly specify value for tagging
Resource schema is valid.
Generated files for MongoDB::Atlas::NetworkPeering
env GOOS=linux CGO_ENABLED=0  GOARCH=amd64 go build -ldflags="-s -w -X 'github.com/mongodb/mongodbatlas-cloudformation-resources/util.defaultLogLevel=debug'" -tags="logging callback metrics scheduler" -o bin/handler cmd/main.go
# github.com/mongodb/mongodbatlas-cloudformation-resources/network-peering/cmd/resource
cmd/resource/resource.go:164:15: currentModel.ProviderName undefined (type *Model has no field or method ProviderName)
cmd/resource/resource.go:301:9: model.ProviderName undefined (type Model has no field or method ProviderName)
make: *** [Makefile:20: debug] Error 2
##[error]Error: The process '/usr/bin/bash' failed with exit code 2
Finishing: Register all custom CFN resources

Additional Context

Installed Go version 1.19.2
Python version: 20.0.2
Ubuntu information
version: Ubuntu 22.04.1 LTS
Linux kernel version: 5.15.0-1022-azure
Image Version: 20221027.1
link to image: https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2204-Readme.md

I had to run the script a couple of times since I got error in versions of packages so in total, I had to add the following to get the correct versions and get further into the script:

pip install --upgrade requests==2.26.0
pip install --upgrade PyYAML==5.4.1
pip install --upgrade MarkupSafe==2.0.1
pip install --upgrade aws-sam-cli
pip install --upgrade Docker==4.3.1
pip install --upgrade Jinja2==2.11.3

Unable to build resources

After pulling a fresh clone of the repo I'm not able to build any resource in order to cfn submit.
Looks like some sdk or go-library change. Here is the error:
(Note I added some debugging log lines to the cfn python to figure out another issue, please disregard that in the following output.)

 mongodbatlas-cloudformation-resources git:(master) ✗ cd project 
➜  project git:(master) ✗ make
cfn generate
prop=/properties/Id
path=('properties', 'Id')
prop=/properties/Created
path=('properties', 'Created')
prop=/properties/ClusterCount
path=('properties', 'ClusterCount')
Generated files for MongoDB::Atlas::Project
env GOOS=linux go build -ldflags="-s -w" -tags="logging callback scheduler" -o bin/handler cmd/main.go
# github.com/mongodb/mongodbatlas-cloudformation-resources/project/cmd/resource
cmd/resource/resource.go:16:73: currentModel.ApiKeys.PublicKey.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:16:115: currentModel.ApiKeys.PrivateKey.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:22:28: currentModel.Name.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:23:29: currentModel.OrgId.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:29:18: cannot use encoding.NewString(project.ID) (type *encoding.String) as type *string in assignment
cmd/resource/resource.go:30:23: cannot use encoding.NewString(project.Created) (type *encoding.String) as type *string in assignment
cmd/resource/resource.go:31:28: cannot use encoding.NewInt(int64(project.ClusterCount)) (type *encoding.Int) as type *int in assignment
cmd/resource/resource.go:42:73: currentModel.ApiKeys.PublicKey.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:42:115: currentModel.ApiKeys.PrivateKey.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:47:24: currentModel.Id.Value undefined (type *string has no field or method Value)
cmd/resource/resource.go:47:24: too many errors
make: *** [Makefile:5: build] Error 2

error getting private connection string from cluster

Hello,

I would like to report an issue when getting the private connection string from cluster (via cluster.connectionStrings.private or privateSrv). The error is thrown if the cluster is just created or the private endpoint has been set up in some previous deploy of the Cloudformation stack.
Error reported by Cloudformation:
Attribute 'ConnectionStrings.Private' does not exist

Sample code:

const cluster = new CfnCluster(scope, `${props.config.app.appName}AtlasCluster`, {
            projectId: props.projectId,
            name: myName,
            mongoDbMajorVersion: '4.4',
        })
const srvAddress = cluster.connectionStrings.private

The srvAddress variable is then exported as Cloudformation stack output.

For reference, from package.json:

...
"dependencies": {
    "@mongodbatlas-awscdk/cluster": "^1.1.5",
    "@mongodbatlas-awscdk/database-user": "^1.1.4",
    "@mongodbatlas-awscdk/network-peering": "^1.1.5",
    "@mongodbatlas-awscdk/private-endpoint": "^1.1.5",
    "@mongodbatlas-awscdk/project": "^1.1.5",
    "@mongodbatlas-awscdk/project-ip-access-list": "^1.1.5",
    ...
},

Thank you for your time and attention to this matter.

Master broken: ProviderName missing from NetworkPeering schema

Current master is broken because of incomplete network-peering schema

Steps to Reproduce

Run cfn-submit-helper.sh network-peering

Expected Behavior

NetworkPeering resource should be registered on Cloudformation.

Actual Behavior

+ cd network-peering
+ echo 'resource: network-peering'
resource: network-peering
+ [[ debug == \d\e\b\u\g ]]
+ make debug
#make -f makebuild  # this runs build steps required by the cfn cli
cfn generate
Explicitly specify value for tagging
Resource schema is valid.
Generated files for MongoDB::Atlas::NetworkPeering
env GOOS=linux CGO_ENABLED=0  GOARCH=amd64 go build -ldflags="-s -w -X 'github.com/mongodb/mongodbatlas-cloudformation-resources/util.defaultLogLevel=debug'" -tags="logging callback metrics scheduler" -o bin/handler cmd/main.go
# github.com/mongodb/mongodbatlas-cloudformation-resources/network-peering/cmd/resource
cmd/resource/resource.go:164:15: currentModel.ProviderName undefined (type *Model has no field or method ProviderName)
cmd/resource/resource.go:301:9: model.ProviderName undefined (type Model has no field or method ProviderName)
make: *** [debug] Error 2

Additional Context

Fixes issues introduced by this commit:
58ef73c

References

Similar to this
#139

getting error while cdk deploy (following cdk example in repo)

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { CfnProject } from '@mongodbatlas-awscdk/project';
import { CfnCluster } from '@mongodbatlas-awscdk/cluster';

interface AtlasStackProps {
  readonly orgId: string;
  readonly profile: string;
  readonly projName: string;
  readonly clusterName: string;
  readonly clusterType: string;
  readonly instanceSize: string;
  readonly region: string;
}

export class Mongo3Stack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const atlasProps = this.getContextProps();

    const projectRes = new CfnProject(this, 'ProjectResource', {
      name: atlasProps.projName,
      orgId: atlasProps.orgId,
      profile: atlasProps.profile
    });

    const clusterRes = new CfnCluster(this, 'ClusterResource', {
      name: atlasProps.clusterName,
      projectId: projectRes.attrId,
      profile: atlasProps.profile,
      clusterType: atlasProps.clusterType,
      backupEnabled: true,
      pitEnabled: false,
      replicationSpecs: [{
        numShards: 1,
        advancedRegionConfigs: [{
          autoScaling: {
            diskGb: {
              enabled: true,
            },
            compute: {
              enabled: false,
              scaleDownEnabled: false,
            },
          },
          analyticsSpecs: {
            ebsVolumeType: "STANDARD",
            instanceSize: atlasProps.instanceSize,
            nodeCount: 3,
          },
          electableSpecs: {
            ebsVolumeType: "STANDARD",
            instanceSize: atlasProps.instanceSize,
            nodeCount: 3,
          },
          readOnlySpecs: {
            ebsVolumeType: "STANDARD",
            instanceSize: atlasProps.instanceSize,
            nodeCount: 3,
          },
          priority: 7,
          regionName: atlasProps.region,
        }]
      }]
    });

  }

  getContextProps(): AtlasStackProps {
    const orgId = this.node.tryGetContext('orgId');
    if (!orgId){
      throw "No context value specified for orgId. Please specify via the cdk context."
    }
    const projName = this.node.tryGetContext('projName') ?? 'test-project-2';
    const profile = this.node.tryGetContext('profile') ?? 'hdahiya2';
    const clusterName = this.node.tryGetContext('clusterName') ?? 'test-cluster';
    const clusterType = this.node.tryGetContext('clusterType') ?? 'REPLICASET';
    const instanceSize = this.node.tryGetContext('instanceSize') ?? "M10";
    const region = this.node.tryGetContext('region') ?? "EU_WEST_1";

    return {
      projName,
      orgId,
      profile,
      clusterName,
      clusterType,
      instanceSize,
      region,
    }
  }
}

The above code which is being referred from example section is getting error while cdk deploy. please help.



[██████████████▌···········································] (1/4)

6:12:40 PM | CREATE_FAILED        | MongoDB::Atlas::Project | ProjectResource
Properties validation failed for resource ProjectResource with message:
#: extraneous key [Profile] is not permitted
6:12:41 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | Mongo3Stack
The following resource(s) failed to create: [ProjectResource, CDKMetadata]. Rollback requested by user.
6:12:41 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | Mongo3Stack
The following resource(s) failed to create: [ProjectResource, CDKMetadata]. Rollback requested by user.
^C

MongoDB Atlast CDK deploy fails with message "internal error".

CloudFormation Template

https://github.com/mongodb/mongodbatlas-cloudformation-resources/tree/master/cfn-resources/cluster/docs

Steps to Reproduce

Deploy a cdk stack with the following configuration:

import { Construct } from 'constructs';
import { 
    aws_cloudformation as cloudformation,
    CfnResource,
    aws_logs as logs,
    CfnTypeActivation
} from 'aws-cdk-lib';

export class Mongo extends Construct {
    constructor(scope: Construct, id: string) {
        super(scope, id);

        const directorRoleArn = "redacted"
        const mongoLogGroup = new logs.LogGroup(this, "mongo_matterbeam_log_group");

        const activateMongoClusterExtension = new cloudformation.CfnTypeActivation(this, `${id}-Cluster`, {
            publicTypeArn: "arn:aws:cloudformation:us-east-1::type/resource/bb989456c78c398a858fef18f2ca1bfc1fbba082/MongoDB-Atlas-Cluster",
            executionRoleArn: "redacted",
            loggingConfig: {
                logGroupName: mongoLogGroup.logGroupName,
                logRoleArn: directorRoleArn 
            }
        });

        const activateMongoDatabaseUserExtension = new cloudformation.CfnTypeActivation(this, `${id}-DatabaseUser`, {
            publicTypeArn: "arn:aws:cloudformation:us-east-1::type/resource/bb989456c78c398a858fef18f2ca1bfc1fbba082/MongoDB-Atlas-DatabaseUser",
            executionRoleArn: "redacted",
            loggingConfig: {
                logGroupName: mongoLogGroup.logGroupName,
                logRoleArn: directorRoleArn 
            }
        });

        const activateMongoNetworkPeeringExtension = new cloudformation.CfnTypeActivation(this, `${id}-NetworkPeering`, {
            publicTypeArn: "arn:aws:cloudformation:us-east-1::type/resource/bb989456c78c398a858fef18f2ca1bfc1fbba082/MongoDB-Atlas-NetworkPeering",
            executionRoleArn: "redacted",
            loggingConfig: {
                logGroupName: mongoLogGroup.logGroupName,
                logRoleArn: directorRoleArn 
            }
        });

        const activateMongoProjectExtension = new cloudformation.CfnTypeActivation(this, `${id}-Project`, {
            publicTypeArn: "arn:aws:cloudformation:us-east-1::type/resource/bb989456c78c398a858fef18f2ca1bfc1fbba082/MongoDB-Atlas-Project",
            executionRoleArn: "redacted",
            loggingConfig: {
                logGroupName: mongoLogGroup.logGroupName,
                logRoleArn: directorRoleArn 
            }
        });

        const activateMongoAwsIamDatabaseUserExtension = new cloudformation.CfnTypeActivation(this, `${id}-AwsIamDatabaseUser`, {
            publicTypeArn: "arn:aws:cloudformation:us-east-1::type/resource/9331cf547939e23b9c7f24086db031317893be87/MongoDb-Atlas-AwsIamDatabaseUser",
            executionRoleArn: "redacted",
            loggingConfig: {
                logGroupName: mongoLogGroup.logGroupName,
                logRoleArn: directorRoleArn 
            }
        });

        const mongo = new CfnResource(this, "mongo_matterbeam", {
            type: "MongoDB::Atlas::Cluster",
            properties: {
                Name: "mongo_matterbeam"
            }
        });
    }
}

Expected Behavior

A MongoDB Atlas cluster should be deployed to my aws stack.

Actual Behavior

The command cdk deploy -v fails due to an Internal Failure.

Here is the full debug log:

CustomerStack | 0/3 | 4:34:15 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack                  | CustomerStack User Initiated
CustomerStack | 0/3 | 4:34:24 PM | CREATE_IN_PROGRESS   | MongoDB::Atlas::Cluster                     | MongoMatterbeam/mongo_matterbeam (MongoMatterbeammongomatterbeam4673F23D) 

CustomerStack | 0/3 | 5:10:32 PM | CREATE_FAILED        | MongoDB::Atlas::Cluster                     | MongoMatterbeam/mongo_matterbeam (MongoMatterbeammongomatterbeam4673F23D) Internal Failure
CustomerStack | 0/3 | 5:10:35 PM | UPDATE_ROLLBACK_IN_P | AWS::CloudFormation::Stack                  | CustomerStack Internal Failure
CustomerStack | 1/3 | 5:10:43 PM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack                  | CustomerStack 
CustomerStack | 2/3 | 5:13:09 PM | DELETE_COMPLETE      | MongoDB::Atlas::Cluster                     | MongoMatterbeam/mongo_matterbeam (MongoMatterbeammongomatterbeam4673F23D) 
CustomerStack | 3/3 | 5:13:09 PM | UPDATE_ROLLBACK_COMP | AWS::CloudFormation::Stack                  | CustomerStack 

Failed resources:
CustomerStack | 5:10:32 PM | CREATE_FAILED        | MongoDB::Atlas::Cluster                     | MongoMatterbeam/mongo_matterbeam (MongoMatterbeammongomatterbeam4673F23D) Internal Failure

 ❌  CustomerStack failed: Error: The stack named CustomerStack failed to deploy: UPDATE_ROLLBACK_COMPLETE: Internal Failure
    at prepareAndExecuteChangeSet (/opt/node-v14.17.6-linux-x64/lib/node_modules/aws-cdk/lib/api/deploy-stack.ts:385:13)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at CdkToolkit.deploy (/opt/node-v14.17.6-linux-x64/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:209:24)
    at initCommandLine (/opt/node-v14.17.6-linux-x64/lib/node_modules/aws-cdk/lib/cli.ts:341:12)
Notices refreshed

The stack named CustomerStack failed to deploy: UPDATE_ROLLBACK_COMPLETE: Internal Failure
Error: The stack named CustomerStack failed to deploy: UPDATE_ROLLBACK_COMPLETE: Internal Failure
    at prepareAndExecuteChangeSet (/opt/node-v14.17.6-linux-x64/lib/node_modules/aws-cdk/lib/api/deploy-stack.ts:385:13)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at CdkToolkit.deploy (/opt/node-v14.17.6-linux-x64/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:209:24)
    at initCommandLine (/opt/node-v14.17.6-linux-x64/lib/node_modules/aws-cdk/lib/cli.ts:341:12)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.