Giter Club home page Giter Club logo

former2's Introduction

Former2

CloudFormation Terraform

Generate CloudFormation / Terraform / Troposphere templates from your existing AWS resources

Screenshot

Screenshot

Overview

Former2 allows you to generate Infrastructure-as-Code outputs from your existing resources within your AWS account. By making the relevant calls using the AWS JavaScript SDK, Former2 will scan across your infrastructure and present you with the list of resources for you to choose which to generate outputs for.

Installation

Though some AWS services do not require it, you will need to install the Former2 Helper browser extension in order to have support for all AWS services. The extension exists to bypass a lack of CORS on some services, such as S3 and IAM.

Install Former2 Helper for Google Chrome

Install Former2 Helper for Mozilla Firefox

Install Former2 Helper for Microsoft Edge

Alternatively, you can download and install the extension yourself.

Usage

Visit former2.com to start.

You will need an IAM key pair to authenticate your requests. If you are not planning on importing resources directly, it is recommended that you provide only read access with these credentials and suggest you assign the ReadOnlyAccess policy. If you intend to use the Import feature, you should grant appropriate permissions to create the stack.

Once authenticated you can navigate via the dashboard or sidebar to specific services, select the resources to add to your outputs and finally click the "Generate" button at the top of the screen.

The following outputs are currently supported:

  • CloudFormation
  • Terraform
  • Troposphere
  • CDK V1 (Cfn Primitives) - TypeScript, Python, Java, C#
  • CDK V2 (Cfn Primitives) - TypeScript, Python, Java, C#
  • CDK for Terraform - TypeScript
  • Pulumi - TypeScript
  • Diagram - embedded version of draw.io

Former2 CLI

A command-line version of Former2 with limited functionality is available for installation. For more information, see the CLI instructions.

LocalStack Support

To enable support for use against LocalStack endpoints, enable the setting in the Settings page, ensure you are using the Former2 Helper extension/add-on, and add the following configuration to LocalStack:

For Google Chrome:

EXTRA_CORS_ALLOWED_ORIGINS=chrome-extension://fhejmeojlbhfhjndnkkleooeejklmigi

For Mozilla Firefox:

EXTRA_CORS_ALLOWED_ORIGINS=moz-extension://853c673f-1bd8-4226-a5ff-f1473f7b3d90

For Microsoft Edge:

EXTRA_CORS_ALLOWED_ORIGINS=extension://okkjnfohglnomdbpimkcdkiojbeiedof

Security

Former2 does not create any resources within your AWS account.

Calls to the AWS service API endpoints are made either directly with the JavaScript SDK or via the browser extension (which also hits endpoints directly). Resource data and your credentials are kept entirely in memory and is never sent over the internet or anywhere else. The credentials are only used to sign requests to AWS endpoints. You should take care to remove any sensitive data (passwords etc.) when sharing your generated code/templates with others.

If you prefer not to use a publicly hosted site to use Former2, you can host your own version by running a HTTP server from the root of the repository. Extension support will also be available if you host on 127.0.0.1 or localhost, otherwise you can modify the extension as needed.

Pricing

Though Former2 is free to access or use locally, some AWS services have small charges associated with API calls so usage may attract an extra couple cents to your AWS bill.

FAQ

Does this replace Console Recorder?

Console Recorder still fills a certain gap and has features this tool does not support (such as Get/List outputs). I'll be maintaining both for as long as I can.

I found an issue / I'm missing an output / Something's wrong. Can you help?

If you find a bug or want to raise a feature request, please do so via the issues page.

former2's People

Contributors

3p3r avatar cairnswm avatar darioackermann avatar dependabot[bot] avatar dnicolson avatar enm10k avatar gruebel avatar iann0036 avatar jairov4 avatar jlandowner avatar nazoking avatar nelg avatar piether avatar rawis avatar reyes256 avatar sean-goodlab avatar sleavitt avatar stefan-matic avatar toshimasa-nanaki avatar vardominator avatar vseryakov avatar whummer avatar yeminiori avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

former2's Issues

Former2 hanging on Chrome

I am trying to scan my Glue resources. Even after a half hour, the I still see the wait circle--the one that comes up before I'm allowed to select the resources to include in the output. I am able to do this for other services fine (I've done IAM and S3, for example).

Escape regex in cloudformation, for backslash

firstly, great tool, thanks for making it.

secondly, an enhancement request. When generating the cloud formation yaml from my API Gateway resource, any regex needs to be properly escaped if it contains the backslash character. Currently it generates an error during import [ import done via Cloud Formation into another AWS account].

original value was in an Authorizer, in the field Token Validation:

^Bearer [-0-9a-zA-z\.]*$

example of former2 generated code [run locally ]

Resources:
  apigateway03da9b1:
    Properties:
      IdentityValidationExpression: "^Bearer [-0-9a-zA-z\.]*$"

manually corrected code, by adding an additional backslash

Resources:
  apigateway03da9b1:
    Properties:
      IdentityValidationExpression: "^Bearer [-0-9a-zA-z\\.]*$"      

doc reference

"Since AWS CloudFormation templates use the JSON syntax for specifying objects and data, you will need to add an additional backslash to any backslash characters in your regular expression, or JSON will interpret these as escape characters."
thanks

Add Dockerfile

Adding aws api keys to any external website is a no no in my opinion.
Running this locally on docker should be the recommended way forward.

Tested with nginx:1.15 and it worked like a charm.

FROM nginx:1.15
COPY . /usr/share/nginx/html

The above is all that's really needed. Port 80 is exposed by default in the container.
Build with docker build -t former2_local:1.0 .
Run the container with docker run --name former2 -p $host_port:80 -d former2_local:1.0

VPC - Network ACL - Can't add

Many of my network ACLs are defined as:

All Traffic - All - All - Source (some ip range).

On the Network ACL Entries tab, I see the NACL ids, but it shows undefined undefined in the CIDR block portion. Even with the ones that have a specific protocol (UDP) and port (500), it does not show up. This is for GovCloud West.

When I expand the details, the CIDR block does show up correctly which is odd. I just can't add them to my template.

VPC - GovCloud Failure

Absolutely love the tool so far. I'm trying to reverse-engineer my VPC in GovCloud (US-Gov-West-1) and it's partially completing because the Client VPN service does not exist yet in GovCloud. That blocks out the main VPC UI because the calls never succeed. I can temporarily unblock the UI by modifying the CSS in the dev toolbar.

Is there a way to fix this behavior of services/features not yet existing in GovCloud?

Generate References instead of absolute ARNs

This is probably going to be way too difficult but I thought I'd throw the idea out there.

In CloudFormation stacks you can use Ref: and Fn::GetAtt to pull dynamic information from other resources. Would it be feasible or even possible to generate the References or attribute fetchers instead of populating with the full ARN?

ElasticBeanstalk CNAME is not a valid URL

In looking at the data that is returned for ElasticBeanstalk environments, the CNAME value is a hostname, not a valid url to be passed to new URL(). When generating output for Beanstalk environments, I got an error Uncaught TypeError: Failed to construct 'URL': Invalid URL. I was able to get around the issue temporarily by prepending the CNAME value with https://, but I don't know if that is the optimal solution.

former2/js/mappings.js

Lines 4027 to 4028 in 2a89424

reqParams.cfn['CNAMEPrefix'] = new URL(obj.data.CNAME).hostname.split(".")[0].split("-").splice(-1, 1).join("-");
reqParams.tf['cname_prefix'] = new URL(obj.data.CNAME).hostname.split(".")[0].split("-").splice(-1, 1).join("-");

`ReferenceError: computeResource is not defined` when using AWS Batch

When trying to generate any format (CFN, TF, etc) from an AWS Batch resource, I get the error: ReferenceError: computeResource is not defined

Attached is the (santised) debug information:

[
    {
        "id": "arn:aws:batch:ap-southeast-2:999999999999:compute-environment/my-silly-compute-environment",
        "type": "batch.computeenvironment",
        "data": {
            "computeEnvironmentName": "my-silly-compute-environment",
            "computeEnvironmentArn": "arn:aws:batch:ap-southeast-2:999999999999:compute-environment/my-silly-compute-environment",
            "ecsClusterArn": "arn:aws:ecs:ap-southeast-2:999999999999:cluster/my-silly-compute-environment_Batch_d03cf48b-4c52-3141-806f-5fd0c006d04d",
            "type": "MANAGED",
            "state": "ENABLED",
            "status": "VALID",
            "statusReason": "ComputeEnvironment Healthy",
            "computeResources": {
                "type": "EC2",
                "minvCpus": 0,
                "maxvCpus": 256,
                "desiredvCpus": 0,
                "instanceTypes": [
                    "optimal"
                ],
                "subnets": [
                    "subnet-deadfa11",
                    "subnet-deadbeef",
                    "subnet-c0ffeeee"
                ],
                "securityGroupIds": [
                    "sg-badbadba"
                ],
                "instanceRole": "arn:aws:iam::999999999999:instance-profile/ecsInstanceRole",
                "tags": {
                    "Name": "myscripts-compute-environment"
                }
            },
            "serviceRole": "arn:aws:iam::999999999999:role/service-role/AWSBatchServiceRole"
        },
        "region": "ap-southeast-2"
    }
]

Error when scanning ecs services

got an error message:
ECS.decripeServices , cluster not found.

The Clusters and the tasks definition are discovered correctly , but got this error message for the Service.
Screen Shot 2019-12-18 at 2 50 21 pm

Error from generated AWS::EC2::Subnet - missing Ipv6CidrBlock

AssignIpv6AddressOnCreation: false was included, but Ipv6CidrBlock was not.

Per https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-subnet.html#cfn-ec2-subnet-ipv6cidrblock:

If you specify AssignIpv6AddressOnCreation, you must also specify Ipv6CidrBlock.

Here's a quick-n-dirty suggestion (not confident enough of your logic to do a PR at this time or I would).

https://raw.githubusercontent.com/iann0036/former2/master/js/mappings.js#3119

                    if ( obj.data.AssignIpv6AddressOnCreation ) {
                        reqParams.cfn['AssignIpv6AddressOnCreation'] = obj.data.AssignIpv6AddressOnCreation;
                        reqParams.tf['assign_ipv6_address_on_creation'] = obj.data.AssignIpv6AddressOnCreation;

                        reqParams.cfn['Ipv6CidrBlock'] = obj.data.Ipv6CidrBlock;
                        reqParams.tf['ipv6-cidr-block'] = obj.data.Ipv6CidrBlock;

                        /* TODO:
                        Tags:
                            - Resource Tag
                        */
                    }

Include referenced resources on add

Add prompt to include referenced resources when adding a single resource to outputs.

e.g. Include a security group when adding an instance

elbv2 TargetGroupName is being left off on output

elbv2 loadbalancer to elb2 listener and elb2 listener rules are clear, But there is absolutely zero connection to the elbv2 target groups.

Raw has the data, but no other format is providing the TargetGroupName

The TargetGroupName is being omitted on the output, this must be there, or there is no way to create a functioning template based on the data, as there is no clear path to the required TargetGroupName, the name must be part of the template output

And actually TargetGroupName is not valid, so it should be just Name, but it should be pushed to the template options.

What is available:

TargetGroupArn: arn:aws:elasticloadbalancing:us-west-1:SECURED:targetgroup/targetgroupname/8f17490f4a037da8
**TargetGroupName: dev03-3048**
Protocol: HTTP
Port: 3048
VpcId: SECURED
HealthCheckProtocol: HTTP
HealthCheckPort: traffic-port
HealthCheckEnabled: true
HealthCheckIntervalSeconds: 5
HealthCheckTimeoutSeconds: 2
HealthyThresholdCount: 2
UnhealthyThresholdCount: 2
HealthCheckPath: /status.html
Matcher:
    HttpCode: 200
LoadBalancerArns:
    - arn:aws:elasticloadbalancing:us-west-1:SECURED:loadbalancer/app/loadbalancername/e874f0d809d66790
TargetType: instance

What is being provided:

    "elbvtargetgroup": {
        "Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
        "Properties": {
            "HealthCheckIntervalSeconds": 30,
            "HealthCheckPath": "/status.html",
            "Port": 80,
            "Protocol": "HTTP",
            "HealthCheckPort": "traffic-port",
            "HealthCheckProtocol": "HTTP",
            "HealthCheckTimeoutSeconds": 5,
            "UnhealthyThresholdCount": 2,
            "TargetType": "instance",
            "Matcher": {
                "HttpCode": "200"
            },
            "HealthyThresholdCount": 2,
            "VpcId": "vpc-SECURED
        }
    },

RAW (has it, so the template formats must be updated to also grab the Name)
"id": "adm-dev-dev05",
"type": "elbv2.targetgroup",
"data": {
"TargetGroupArn": "arn:aws:elasticloadbalancing:us-west-1:SECURED:targetgroup/adm-dev-dev05/8cfa129b71b647da",
"TargetGroupName": "adm-dev-dev05", <--Should be "Name":
"Protocol": "HTTP",
"Port": 80,
"VpcId": "vpc-c02860a5",
"HealthCheckProtocol": "HTTP",
"HealthCheckPort": "traffic-port",
"HealthCheckEnabled": true,
"HealthCheckIntervalSeconds": 30,
"HealthCheckTimeoutSeconds": 5,
"HealthyThresholdCount": 5,
"UnhealthyThresholdCount": 2,
"HealthCheckPath": "/status.html",
"Matcher": {
"HttpCode": "200"
},
"LoadBalancerArns": [
"arn:aws:elasticloadbalancing:us-west-1:SECURED:loadbalancer/app/loadbalancer/e874f0d809d66790"
],
"TargetType": "instance"
},
"region": "us-west-1"
},

Rate Exceeded on AWS API

Hi,

I am currently having issues with the rate limit on AWS side due to a high number of resources on my account.
I am running the container locally but I do have the same issue with the version hosted on https://former2.com.

Env:
macOS 10.14.4
Docker version 18.09.2, build 6247962
Image build on former2 commit 5d8ea581530dda7272e391ded3eaa7c00f6cbe53
Firefox 66.0.5 with former2 extension installed

The issue happens when launching the scan.
eg. of requests

url: https://ecs.eu-west-1.amazonaws.com/
{"taskDefinition":"arn:aws:ecs:eu-west-1:<accountid>:task-definition/<taskdefinition>:<version>"}

This ends up with the following error:

HTTP/1.1 400 Bad Request
x-amzn-RequestId: 6af5fd30-765e-11e9-a5f9-b7e8a953a005
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: x-amzn-RequestId,x-amzn-ErrorType,x-amzn-ErrorMessage,Date
Content-Type: application/x-amz-json-1.1
Content-Length: 58
Date: Tue, 14 May 2019 15:39:10 GMT
Connection: close

{"__type":"ThrottlingException","message":"Rate exceeded"}

Could we add either a mecanism to retry on rate limited error or a rate limit setting in the configuration?

Thanks!

generate CLI deletion of all selected resources

Hi,

As a feature request / enhancement, it would be awesome if former2 could generate AWS CLI output to delete all selected resources. For people with Lab accounts, they often end up having AWS resources created which should be cleaned up, but with no clear way to delete everything within an AWS account. VPC deletion is OK, but misses many other types of resources.

It looks like Former2, with it's ability to scan many of the resources could maybe also output the AWS CLI syntax to just delete all the resources selected.

"Add Selected" button does nothing

I have entered credentials, navigated to "Migration & Transfer" then hit refresh.
Resources appear however clicking the checkbox then clicking "Add Selected" does nothing.

No errors in javascript console.

Environment, MountPoints, PortMappings, VolumesFrom not picked up on AWS::ECS::TaskDefinition

Hi Ian

When I generate a cloudformation template for a task definition these properties are not picked up. From the code I get that they are not filled in the data structures, the call is made to the API to pick them up. Would it be an option to add them ?

Now I get something like this :

    ecs7f93d16:
        Type: "AWS::ECS::TaskDefinition"
        Properties:
            ContainerDefinitions: 
              - 
                Cpu: 0
                Environment: {}
                Essential: true
                Image: "612516126697.dkr.ecr.eu-central-1.amazonaws.com/mn-jenkins"
                Memory: 512
                MountPoints: {}
                Name: "CD-Jenkins"
                PortMappings: {}
                Privileged: true
                User: "root"
                VolumesFrom: {}
            Family: "Task-Jenkins"
            TaskRoleArn: "arn:aws:iam::612516126697:role/ecsTaskExecutionRole"
            ExecutionRoleArn: "arn:aws:iam::612516126697:role/ecsTaskExecutionRole"
            NetworkMode: "bridge"
            Volumes: 
              - 
                Name: "CD-jenkins"
                DockerVolumeConfiguration: 
                    Scope: "task"
                    Driver: "local"
                    DriverOpts: 
                        device: "fs-3af18a63.efs.eu-central-1.amazonaws.com:/"
                        o: "addr=fs-3af18a63.efs.eu-central-1.amazonaws.com,nfsvers=4.0,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2"
                        type: "nfs"
            RequiresCompatibilities: 
              - "EC2"

It would be a great enhancement that would save a lot of time when extracting cloudformation form an environment.

Best
Bart

Incorrect types for some VPC resource properties

I have cfn-lint installed for VSCode and after loading the template that former2 generated the following warnings were shown:

E3012: Property Resources/CustomerGateway/Properties/BgpAsn should be of type Integer
E3012: Property Resources/ec2b729b8b/Properties/Protocol should be of type Integer
E3034: Value has to be between 1 and 32766 at Resources/ec29e8c64b/Properties/RuleNumber

Relevant snippet of template:

ec29e8c64b:
        Type: "AWS::EC2::NetworkAclEntry"
        Properties:
            CidrBlock: "0.0.0.0/0"
            Egress: true
            NetworkAclId: !Ref NetworkAcl1
            Protocol: "-1"
            RuleAction: "deny"
            RuleNumber: 32767

Scanning IAM role produces TypeError: Cannot read property 'replace' of undefined

Hi Ian
When scanning an IAM role I get this error.

When I replace the code in mappings.js at line 51 with this code it works. But I dont know if something essential is then missing.
e = function(a) { if (a) { a = a.replace(/\r\n/g, "\n"); for (var b = "", d = 0; d < a.length; d++) { var c = a.charCodeAt(d); 128 > c ? b += String.fromCharCode(c) : (127 < c && 2048 > c ? b += String.fromCharCode(c >> 6 | 192) : (b += String.fromCharCode(c >> 12 | 224), b += String.fromCharCode(c >> 6 & 63 | 128)), b += String.fromCharCode(c & 63 | 128)) } return b } else { return "";} }(e);

Nothing is added to the output. With this quick fix I get this debug log.

[ { "id": "arn:aws:iam::612516126697:role/rds-monitoring-role", "type": "iam.role", "data": { "Path": "/", "RoleName": "rds-monitoring-role", "RoleId": "AROAY5HGOL7U5DB7TWXKZ", "Arn": "arn:aws:iam::612516126697:role/rds-monitoring-role", "CreateDate": "2019-05-20T14:10:57.000Z", "AssumeRolePolicyDocument": "%7B%22Version%22%3A%222012-10-17%22%2C%22Statement%22%3A%5B%7B%22Sid%22%3A%22%22%2C%22Effect%22%3A%22Allow%22%2C%22Principal%22%3A%7B%22Service%22%3A%22monitoring.rds.amazonaws.com%22%7D%2C%22Action%22%3A%22sts%3AAssumeRole%22%7D%5D%7D", "MaxSessionDuration": 3600, "Tags": [] }, "region": "eu-central-1" } ]

Best
Bart

elbv2 listenerrules : default not an integer. Don't include it. It's in the listener already

Priority: default, is not an integer and in fact it's 100% incorrect in terms of a listenerrule. The default is handled inside the listener and should not be created on output.

Listener:
"elbv27da68c4": {
"Type": "AWS::ElasticLoadBalancingV2::Listener",
"Properties": {
"LoadBalancerArn": {some ELB },
"Port": 80,
"Protocol": "HTTP",
"DefaultActions": [
{
"Order": 1,
"TargetGroupArn": {some target group},
"Type": "forward"

}

Listenerrule below is redundant and in fact will fail, if attempted to use. Best to eliminate this from any and all outputs.

"elbv2d009637": {
"Type": "AWS::ElasticLoadBalancingV2::ListenerRule",
"Properties": {
"Priority": "default",
"ListenerArn": {some listener arn},
"Actions": [
{
"Type": "forward",
"TargetGroupArn": {

UsagePlan output failing to parse CloudFormation

AWS::ApiGateway::UsagePlan.Properties.ApiStages[].Throttle is provided as the following YAML which fails to parse in the CloudFormation Editor with: Cannot convert the template because of an error:: unidentified alias "/*:"

Provided and fails:

                Throttle: 
                    */*: 
                        BurstLimit: 240
                        RateLimit: 120

Edited and works:

                Throttle: 
                    BurstLimit: 240
                    RateLimit: 120

If I remove the */*: line and indent the Limit values accordingly, it parses fine.

AWS::CloudFront::Distribution CustomErrorResponse if ResponsePagePath and ResponseCode are null, don't include cloudformation output

Example

               "CustomErrorResponses": [
                    {
                        "ErrorCode": 404,
                        "ResponsePagePath": "",
                        "ResponseCode": "",
                        "ErrorCachingMinTTL": 20

AWS will reject it, it will not accept null here and expects an integer, This one had nothing configured, so there needs to be some logic to dictate when it's included or not.

{/DistributionConfig/CustomErrorResponses/0/ResponseCode} does not match type {Integer}, Value of property {/DistributionConfig/CustomErrorResponses/1/ResponseCode} does not match type {Integer}, Value of property

AWS::CloudFront::Distribution DistributionConfig setting HttpVersion" HTTP2 != http2 cloudformation output

            "DistributionConfig": {
                **"HttpVersion": "HTTP2",**
                "IPV6Enabled": true

Value 'HTTP2' at 'distributionConfigWithTags.distributionConfig.httpVersion' failed to satisfy constraint: Member must satisfy enum value set: [http2, http1.1]

HttpVersion:
Type: String
Allowed Values: http1.1 | http2

Let me know if these are helpful, sometimes I am not sure what I should bubble up for you all to fix..

Try to scan account getting the following error

Error calling EC2.searchTransitGatewayRoutes
Missing required key 'Filters' in params

I am trying to scan the entire account and this error shows up, I am unsure what to add in the parameters section to avoid this error.

Thanks

Rd

Back-off mechanism is not exponential

I have a rather large AWS account for which I needed to wait a number of minutes before the UI became usable.

When looking at the console I saw a number of lines
Too many requests, sleeping for X ms where X would be less than 1000.
And then a whole lot of lines saying
Too many requests, sleeping for 120000 ms

This is not an exponential back-off, it's a rather drastic back off. It may also (partially) cause the behaviour seen in #2

Former hanging

Hello,

Thank you for this wonderful tool. I am trying to get some detail in CF about IAM and SES and also SQS, but it shows the list of queues in SQS for example but it is greyed out, I cannot do anything else.
Please assist.

I see logs I have added as an attachment

I am using chrome, I tried adding the former extension and then disabling it, and neither works...

I tried firefox and same thing, I also 2 different computers.

f2
former2.com-1576028259710.log

I am guessing that apart of the below message means that I need to add get queue attribute permissions?

Error calling SQS.getQueueAttributes. Access to the resource https://sqs.us-east-1.amazonaws.com/ is denied. datatables.js:260 AccessDenied: Access to the resource https://sqs.us-east-1.amazonaws.com/ is denied.

For IAM I see the following, not sure what happened, it used to all work for IAM

former2.com-2576028683490.log

using former2 programmatically

Hi!

Thanks for this fantastic tool. I have a question. I need to use Former2 programmatically. What's the best way to accomplish this beside using something like puppeteer?

Thanks!

AWS::RDS::DBCluster ScalingConfiguration doesn't support TimeoutAction

I've exported a DBCluster which includes the following ScalingConfiguration that throws an error because TimeoutAction isn't supported.

The template snippet of interest is

            ScalingConfiguration: 
                MinCapacity: 8
                MaxCapacity: 256
                AutoPause: false
                SecondsUntilAutoPause: 3600
                TimeoutAction: "RollbackCapacityChange"

It is not supported in ScalingConfiguration per this doc: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-rds-dbcluster-scalingconfiguration.html

I do see that option with the create-db-cluster CLI command: https://docs.aws.amazon.com/cli/latest/reference/rds/create-db-cluster.html

I can also see that Ruby's API supports it: https://docs.aws.amazon.com/sdkforruby/api/Aws/RDS/Types/ScalingConfigurationInfo.html

I don't see any sign of that option being supported through CloudFormation at all right now.

Logical resource naming within the template

I created a template for our VPC using Former2.

I added all related resources I could think of as well as those suggested by Former2.

This resulted in a large template, over 500 lines long and a lot of resources.

Because all the resources have random logical names it can make identifying them difficult when looking at the template. Although I understand that you could not possibly infer the logical name with anything meaningful, I think it would be useful to put the type of the resource somewhere in the name to at least give some semantic meaning to the resources.

e..g. instead of a resource being called ec26de3980, call it VPCGatewayAttachmentec26de3980

This would help human comprehension of the template, especially when looking at what REF and GetAtt are referring to.

Thanks!

Former2 doesn't do pagination correctly

When looking for a large number of CloudFront or Lambda resources (>100), the list stops at the first 100 and does not show later resources.

In js/datatables.js in sdkcall there is code to handle pagination but the data returned from the AWS SDK was not in a matching scheme.
E.g. it was a layer down like
data = {'DistributionList' = [ 'NextMarker'=xxx, 'Lists'= #etc
and not a peer to DistributionList like
data = {'NextMarker'=xxx, 'DistributionList': # etc

I patched this locally in the browser by changing the function sdkcall as follows:


function sdkcall(svc, method, params, alert_on_errors, backoff) {
    return new Promise(function(resolve, reject) {
        var service = new AWS[svc]({region: region});
        if (svc == "GlobalAccelerator") {
            service = new AWS[svc]({region: 'us-west-2'});
        }

        service[method].call(service, params, async function(err, data) {
            if (err) {
                if (err.code == "TooManyRequestsException" || err.message == "Too Many Requests" || err.code == "ThrottlingException" || err.message == "Rate exceeded" || err.code == "TimeoutError") {
                    if (backoff) {
                        console.log("Too many requests, sleeping for " + backoff + "ms");
                        await new Promise(resolve => setTimeout(resolve, backoff));
                        backoff *= 2;
                    } else {
                        console.log("Too many requests, sleeping for 500ms");
                        await new Promise(resolve => setTimeout(resolve, 500));
                        backoff = 500 + Math.floor(Math.random() * 500);
                    }
                    sdkcall(svc, method, params, alert_on_errors, backoff).then(newdata => {
                        resolve(newdata);
                    }, data => {
                        reject(data);
                    });
                } else {
                    if (err.code == "NetworkingError") {
                        console.log("Skipping " + svc + "." + method + " NetworkingError");
                    } else if (err.code == "AccessDeniedException") {
                        console.log("Skipping " + svc + "." + method + " AccessDeniedException");
                    } else if (err.code == "UnknownError" && svc == "MediaStore") {
                        console.log("Skipping " + svc + "." + method + " UnknownError");
                    } else if (err.code == "ForbiddenException" && svc == "RoboMaker") {
                        console.log("Skipping " + svc + "." + method + " ForbiddenException");
                    } else if (err.code == "AccessDeniedException" && svc == "FSx") {
                        console.log("Skipping " + svc + "." + method + " AccessDeniedException");
                    } else if (alert_on_errors) {
                        console.log("Error calling " + svc + "." + method + ". " + (err.message || JSON.stringify(err)));
                        console.trace(err);
                        $.notify({
                            icon: 'font-icon font-icon-warning',
                            title: '<strong>Error calling ' + svc + '.' + method + '</strong>',
                            message: err.message || JSON.stringify(err)
                        },{
                            type: 'danger'
                        });
                    }
                    
                    reject(data);
                }
            } else {
                 object_keys_zero = Object.keys(data)[0];
                 console.log("xxx: " + object_keys_zero);
                if (data.Marker) {
                    params['Marker'] = data.Marker;
                    sdkcall(svc, method, params, alert_on_errors).then(newdata => {
                        var mergeddata = deepmerge.all([data, newdata]);
                        
                        resolve(mergeddata);
                    }, data => {
                        reject(data);
                    });
                } else if (data.NextPageToken) {
                    params['PageToken'] = data.NextPageToken;
                    sdkcall(svc, method, params, alert_on_errors).then(newdata => {
                        var mergeddata = deepmerge.all([data, newdata]);
                        
                        resolve(mergeddata);
                    }, data => {
                        reject(data);
                    });
                } else if (data.ContinuationToken) {
                    params['ContinuationToken'] = data.ContinuationToken;
                    sdkcall(svc, method, params, alert_on_errors).then(newdata => {
                        var mergeddata = deepmerge.all([data, newdata]);
                        
                        resolve(mergeddata);
                    }, data => {
                        reject(data);
                    });
                } else if (data.NextToken) {
                    params['NextToken'] = data.NextToken;
                    sdkcall(svc, method, params, alert_on_errors).then(newdata => {
                        var mergeddata = deepmerge.all([data, newdata]);
                        
                        resolve(mergeddata);
                    }, data => {
                        reject(data);
                    });
                } else if (data.nextToken) {
                    params['nextToken'] = data.nextToken;
                    sdkcall(svc, method, params, alert_on_errors).then(newdata => {
                        var mergeddata = deepmerge.all([data, newdata]);
                        
                        resolve(mergeddata);
                    }, data => {
                        reject(data);
                    });
                } else if (data[object_keys_zero].NextMarker) {
                    params['Marker'] = data[object_keys_zero].NextMarker;
                    sdkcall(svc, method, params, alert_on_errors).then(newdata => {
                        var mergeddata = deepmerge.all([data, newdata]);
                        
                        resolve(mergeddata);
                    }, data => {
                        reject(data);
                    });

                } else {

                    resolve(data);
                }
            }
        });
    });
}

Add support for AWS profiles in CLI

Former2 was already incredibly useful, thanks for making it, and a CLI makes it even better.

Would you consider adding support for named profiles like the to ease use against multiple accounts? e.g.

former2 --profile development ...
former2 --profile production ...

Performance issues

I love this tool, it is something that AWS should have built in to the console and I am extremely grateful for it.

Unfortunately I am having a lot of performance issues using it.

I would estimate I have only managed to complete a full scan a couple of times, the other times the Chrome process hangs with an unresponsive dialog and the progress counter stops. The last time it was at item 34/102.

On occasion it comes back to life and very slowly moves forward until it stops again.
It rarely completes before I have to close Chrome completely.

I have also tried Firefox but have had similar results.

Is there anything I can do to further diagnose or improve the performance?

Thanks

image

Issue on Cognito section

When clicking on Cognito section, there is an error in the console:
datatables.js:23983 Uncaught (in promise) TypeError: data.Roles.join is not a function at datatables.js:23983 at async Promise.all (/index 0) at async Promise.all (/index 0) at /async http:/127.0.0.1/js/datatables.js:23972

And then, whatever the sub-level (user pool, user client...), nothing happens anymore.

Support for multiple accounts

We have multiple AWS accounts and at the moment it seems switching between the accounts in Former2 means updating credentials and rescanning.

It would be great to have the concept of a profile that you can login to and then switch between accounts. Understandable you may not want to store credentials in your system therefore could former 2 be made an app to run locally?

ECR for terraform drops "name" attribute from aws_ecr_repository

CloudFormation

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
Generator: "former2"
Description: ""
Resources:
ecrcab9c80:
Type: "AWS::ECR::Repository"
Properties:
RepositoryName: "test-api"
name: "test-api"
LifecyclePolicy:
RegistryId: "583566476015"

ecsdc66eaf:
    Type: "AWS::ECS::Cluster"
    Properties:
        ClusterName: "test-api-staging"

==========================================
Same output when Terraform is selected shows up as

provider "aws" {
region = "us-east-1"
}

resource "aws_ecr_repository" "ecrcab9c80" {}

resource "aws_ecs_cluster" "ecsdc66eaf" {
name = "test-api-staging"
}

access_logs.s3.bucket' cannot be empty

Created a template from an ALBv2.

It created the following block for the LoadBalancerAttributes: (note I had not changed any of these values from the defaults)

      LoadBalancerAttributes: 
        - 
          Key: "access_logs.s3.enabled"
          Value: "false"
        - 
          Key: "access_logs.s3.bucket"
          Value: ""
        - 
          Key: "access_logs.s3.prefix"
          Value: ""
        - 
          Key: "idle_timeout.timeout_seconds"
          Value: "60"
        - 
          Key: "deletion_protection.enabled"
          Value: "false"
        - 
          Key: "routing.http2.enabled"
          Value: "true"
        - 
          Key: "routing.http.drop_invalid_header_fields.enabled"
          Value: "false"

I then put this into a template and launched a stack which failed with the following:

The value of 'access_logs.s3.bucket' cannot be empty (Service: AmazonElasticLoadBalancingV2; Status Code: 400; Error Code: ValidationError;

gp2 volumes bring 'iops' setting incorrectly.

The AWS API shows the "iops" setting for gp2 volumes, which of course is a calculation and not a user settable value. When I imported some EC2 volumes, I got the "Iops" value in Cloudformation, which is invalid as they are gp2 types.

I also got SnapshotId values of null, when the key/value pair should be omitted for volumes not being created from a snapshot.

  Volume:
    Type: "AWS::EC2::Volume"
    Properties:
      Iops: 600
      Size: 200
      VolumeType: "gp2"
      SnapshotId: ""

Errors reading from S3

Getting errors in chrome when trying to load s3:

Uncaught (in promise) null
datatables.js:203 Skipping S3.listBuckets NetworkingError
datatables.js:5550 Uncaught (in promise) null
chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71 Uncaught (in promise) Error: Extension context invalidated.
at chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71
at Object.makeRequest (chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71)
at d (chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71)
at Object.sendRequest (chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71)
at Object.requestFunction (chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71)
at Object.callRepository (chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71)
at chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71
at new Promise ()
at Object.t.callBackgroundRepository (chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71)
at e.setFrameReady (chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/onloadwff.js:71)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.