gsa / grace-inventory Goto Github PK

Lambda function to create an inventory report of AWS services as an Excel spreadsheet in an S3 bucket. Includes Terraform code to deploy it.

License: Other

Makefile 1.77% Go 93.38% HCL 4.85%

grace

grace-inventory's Issues

Bucket region issue when Lambda triggers

When the Lambda triggers it looks like I am getting an error due to the region I deployed my infrastructure.

failed to upload report to bucket: AuthorizationHeaderMalformed: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-east-2' status code: 400, request id: BC04B1B8AED32969, host id: Cx3IjHtCIrQnS0wX

I set everything to build in us-east-2. I am going to see if I cannot correct this by moving my S3 bucket for inventory to us-east-1 as I suspect that is the issue.

Run Time Error

panic: runtime error: invalid memory address or nil pointer dereference
/Users/fitzwaterb/go/src/github.com/GSA/grace-inventory/handler/helpers/accounts/accounts.go:64 +0x31

Use Organizations.ListAccountsPages()

Use Organizations.ListAccountsPages() instead of Organizations.ListAccounts in listAccountsForMaster()

Reduce cyclomatic complexity functions

spreadsheet/spreadsheet.go:166:1: cyclomatic complexity 15 of func `(*Sheet).setCell` is high (> 10) (gocyclo)
func (s *Sheet) setCell(cell *xlsx.Cell, val interface{}) {
^
inv/inv.go:294:1: cyclomatic complexity 30 of func `typeToSheet` is high (> 10) (gocyclo)
func typeToSheet(items interface{}) (string, error) {
^
inv/inv.go:242:1: cyclomatic complexity 12 of func `(*Inv).aggregate` is high (> 10) (gocyclo)
func (inv *Inv) aggregate() error {
^

Requires logging S3 bucket

The S3 bucket created for the inventory spreadsheets has logging enabled and requires a pre-existing bucket with a name in the form of ${var.project_name}-${var.appenv}-access-logs. The LogDelivery group must have WRITE and READ_ACP permissions on the bucket (acl = "log-delivery-write"). Add this information to the pre-requisites.

Improve handler/helpers/accounts/accounts_test.go: TestAccountsList()

Use assertions
Use table based testing

Disable fewer linters

Currently have several linters disabled:

  disable:
    - dupl
    - funlen
    - prealloc
    - gochecknoinits
    - gochecknoglobals
    - wsl

Should enable these and fix the issues with the code that required them to be disabled.

How to increase verbosity?

I've implemented the module as shown in the examples (using 'self' as well as within the organization from a master account). All resources have been created successfully, but when I invoke the Lambda function either manually or via Cron scheduler, no report is created.

An error message is shown in the Cloudwatch logs, but nothing I can really work with. Is there a way to increase the verbosity and/ or enable DEBUG information?

START RequestId: 8771c388-99c9-4df8-bfd0-c6c456ef25be Version: $LATEST
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryAccounts
2020/02/14 09:25:35 github.com/GSA/grace-inventory/handler/inv.(*Inv).queryAccounts took 176.360813ms
2020/02/14 09:25:35 Sheet "Accounts" has completed
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryLoadBalancers
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).querySecurityGroups
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryDBSnapshots
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).querySubscriptions
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryParameters
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryInstances
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryKeyPairs
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryConfigRules
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).querySnapshots
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).querySubnets
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryVpcs
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryAddresses
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryKeys
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryImages
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryRoles
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).querySecrets
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryTopics
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryUsers
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryVaults
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryDBInstances
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryBuckets
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryGroups
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryPolicies
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryVolumes
2020/02/14 09:25:35 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryAlarms
2020/02/14 09:25:36 calling github.com/GSA/grace-inventory/handler/inv.(*Inv).queryStacks
2020/02/14 09:25:36 github.com/GSA/grace-inventory/handler/inv.(*Inv).queryRoles took 981.014396ms
2020/02/14 09:25:36 Sheet "Roles" has completed
2020/02/14 09:25:36 github.com/GSA/grace-inventory/handler/inv.(*Inv).queryLoadBalancers took 1.143413201s
unknown type: errorString
null
2020/02/14 09:25:37 Unknown sheet type: *elbv2.LoadBalancer
END RequestId: 8771c388-99c9-4df8-bfd0-c6c456ef25be
REPORT RequestId: 8771c388-99c9-4df8-bfd0-c6c456ef25be	Duration: 2535.63 ms	Billed Duration: 2600 ms	Memory Size: 128 MB	Max Memory Used: 74 MB	Init Duration: 167.55 ms

Terraform config - data.aws_caller_identity.current.account_id not enclosed properly

File: main.tf
Line: 6

The reference for id from the data source aws_caller_identity is not properly enclosed in "${}".
Is currently:
account_id = data.aws_caller_identity.current.account_id
Should be:
account_id = "${data.aws_caller_identity.current.account_id}"

Error thrown:

Error: MalformedPolicyDocumentException: Policy contains a statement with one or more invalid principals.
	status code: 400, request id: 2425f0db-3033-448c-8e20-347eec8cac03

  on .terraform/modules/example_self/kms.tf line 1, in resource "aws_kms_key" "kms_key":
   1: resource "aws_kms_key" "kms_key" {

Error setting resource tags

Looks like some circular dependency

Error: error updating S3 Bucket (test123-development-inventory) tags: error setting resource tags (test123-development-inventory): NoSuchBucket: The specified bucket does not exist
status code: 404, request id: 945A07546A79C49D, host id: J5HUtj/dTQ33E3IzpXE5nyuIxkhSi3XKkRB52EQYkIiPV3Yn+axT01TVOi/elJzy9uJPVz5kOz8=

on .terraform/modules/grace_inventory/s3.tf line 1, in resource "aws_s3_bucket" "bucket":
1: resource "aws_s3_bucket" "bucket" {

typeToSheet should contain all supported types

As noted in Issue #38 a bug was identified where *elbv2.LoadBalancer did not exist in the typeToSheet method causing report generation failure.

All known and supported types should be present in typeToSheet, I have compiled a quick list of items not covered by the typeToSheet method. As the method is being resolved, each type should likely have a test for typeToSheet that verifies at minimum there is no error.

Sheet	Type	Missing
SheetRoles	*iam.Role	False
SheetGroups	*iam.Group	False
SheetPolicies	*iam.Policy	False
SheetUsers	*iam.User	False
SheetBuckets	*s3.Bucket	False
SheetInstances	*ec2.Instance	False
SheetImages	*ec2.Image	False
SheetVolumes	*ec2.Volume	False
SheetSnapshots	*ec2.Snapshot	False
SheetVpcs	*ec2.Vpc	False
SheetSubnets	*ec2.Subnet	False
SheetSecurityGroups	*ec2.SecurityGroup	False
SheetAddresses	*ec2.Address	True
SheetKeyPairs	*ec2.KeyPairInfo	True
SheetStacks	*cloudformation.Stack	True
SheetAlarms	*cloudwatch.MetricAlarm	True
SheetConfigRules	*configservice.ConfigRule	False
SheetLoadBalancers	*elbv2.LoadBalancer	True
SheetVaults	*glacier.DescribeVaultOutput	False
SheetKeys	*helpers.KmsKey	False
SheetDBInstances	*rds.DBInstance	False
SheetDBSnapshots	*rds.DBSnapshot	False
SheetSecrets	*secretsmanager.SecretListEntry	False
SheetSubscriptions	*sns.Subscription	False
SheetTopics	*helpers.SnsTopic	False
SheetParameters	*ssm.ParameterMetadata	False

Make list of sheets configurable

Make the list of sheets configurable:

grace-inventory/handler/main.go

Lines 21 to 48 in 1270dbe

 sheets := []string{ 

 inv.SheetAccounts, 

 inv.SheetBuckets, 

 inv.SheetGroups, 

 inv.SheetImages, 

 inv.SheetInstances, 

 inv.SheetPolicies, 

 inv.SheetRoles, 

 inv.SheetSecurityGroups, 

 inv.SheetSnapshots, 

 inv.SheetSubnets, 

 inv.SheetUsers, 

 inv.SheetVolumes, 

 inv.SheetVpcs, 

 inv.SheetAddresses, 

 inv.SheetKeyPairs, 

 inv.SheetStacks, 

 inv.SheetAlarms, 

 inv.SheetConfigRules, 

 inv.SheetLoadBalancers, 

 inv.SheetVaults, 

 inv.SheetKeys, 

 inv.SheetDBInstances, 

 inv.SheetDBSnapshots, 

 inv.SheetSecrets, 

 inv.SheetSubscriptions, 

 inv.SheetTopics, 

 inv.SheetParameters,

	sheets := []string{
	inv.SheetAccounts,
	inv.SheetBuckets,
	inv.SheetGroups,
	inv.SheetImages,
	inv.SheetInstances,
	inv.SheetPolicies,
	inv.SheetRoles,
	inv.SheetSecurityGroups,
	inv.SheetSnapshots,
	inv.SheetSubnets,
	inv.SheetUsers,
	inv.SheetVolumes,
	inv.SheetVpcs,
	inv.SheetAddresses,
	inv.SheetKeyPairs,
	inv.SheetStacks,
	inv.SheetAlarms,
	inv.SheetConfigRules,
	inv.SheetLoadBalancers,
	inv.SheetVaults,
	inv.SheetKeys,
	inv.SheetDBInstances,
	inv.SheetDBSnapshots,
	inv.SheetSecrets,
	inv.SheetSubscriptions,
	inv.SheetTopics,
	inv.SheetParameters,

gsa / grace-inventory Goto Github PK

grace-inventory's Issues

Recommend Projects

Recommend Topics

Recommend Org