logzio / logzio_terraform_client Goto Github PK

coverage at 0.2.1 is under 80%

task complete when coverage >= 85%

needs to be a param, as it's specifiable per sub-account user created

the client creation for alerts and endpoints needs to be tighter and throw an error if not done correctly

won't do as part of this issue

issue completed by moving out the API endpoint into something specified by the client of this client....

documentation : update readme with endpoints docs - seems to have alerts and users but not endpoints for some reason - maybe move this lot into godoc...

endpoints aren't all be cleaned up after testing

travis ci - refuses to run integration tests consistently

probably due to dodgy memory management inside the tests...

update alert function has an http method of DELETE

Currently, the tests seem to require a valid API token and is actually making requests against the Logz.io API. This makes it non-trivial to contribute to the project as it requires to have an Enterprise account just for running the tests. I suggest to use a mock for the tests and complement with integration tests which can do real requests against the API if desired.

custom endpoints
header - internally this is a string, externally this should be a map[string]string
bodytemplate - internally this is a json object, externally, this should be an interface{}

Test code:

func Test(apiToken string, baseUrl string, subAccountNumber int64) error {
    var c *sub_accounts.SubAccountClient
	if len(apiToken) == 0 {
		fmt.Errorf("API token not defined")
	}
	if len(baseUrl) == 0 {
		fmt.Errorf("Base URL not defined")
	}
    c, _ =  sub_accounts.New(apiToken, baseUrl)

    subAccount, err := c.GetSubAccount(subAccountNumber)
	if err != nil {
		return err
	}
    fmt.Println("%s", subAccount.AccountName)
    return nil
}

Result:

panic: interface conversion: interface {} is nil, not float64

Root cause:
Not checking nil value when accessing json['key'] in function jsonToSubAccount (specifically in the field MaxDailyGB)
See the code below:

func jsonToSubAccount(json map[string]interface{}) SubAccount {
 
	subAccount := SubAccount{
		Id:                    int64(json[fldAccountId].(float64)),
		AccountName:           json[fldAccountName].(string),
		MaxDailyGB:            float32(json[fldMaxDailyGB].(float64)),  <---------- BUG ------------
		RetentionDays:         int32(json[fldRetentionDays].(float64)),
		Searchable:            json[fldSearchable].(bool),
		Accessible:            json[fldAccessible].(bool),
		DocSizeSetting:        json[fldDocSizeSetting].(bool),
		SharingObjectAccounts: json[fldSharingAccountObjects].([]interface{}),
		UtilizationSettings:   json[fldUtilizationSettings].(map[string]interface{}),
	}
....
}

Debug log:

{
	"accessible": true,
	"accountId": 12345,
	"accountName": "xxxxx",
	"docSizeSetting": true,
	"email": null,
	"isFlexible": true,
	"maxDailyGB": null,  <------------- The returned value ------------ 
	"reservedDailyGB": 0,
	"retentionDays": 14,
	"searchable": true,
	"sharingObjectsAccounts": [],
	"utilizationSettings": {
		"frequencyMinutes": 60,
		"utilizationEnabled": true
	}
}

The fix:
I was not able to push code even to side branch !
For all field need to validate values are NOT nil (in case of nil set some default):

func jsonToSubAccount(json map[string]interface{}) SubAccount {
	var maxDailyGB float32 = 0
	if json[fldMaxDailyGB] != nil {
		maxDailyGB = float32(json[fldMaxDailyGB].(float64))
	}
	subAccount := SubAccount{
		Id:                    int64(json[fldAccountId].(float64)),
		AccountName:           json[fldAccountName].(string),
		MaxDailyGB:            maxDailyGB,
		RetentionDays:         int32(json[fldRetentionDays].(float64)),
		Searchable:            json[fldSearchable].(bool),
		Accessible:            json[fldAccessible].(bool),
		DocSizeSetting:        json[fldDocSizeSetting].(bool),
		SharingObjectAccounts: json[fldSharingAccountObjects].([]interface{}),
		UtilizationSettings:   json[fldUtilizationSettings].(map[string]interface{}),
	}

		if json[fldUtilizationSettings] != nil {
			subAccount.UtilizationSettings = json[fldUtilizationSettings].(map[string]interface{})
			for key, value := range subAccount.UtilizationSettings {
				if value == nil {
					delete(subAccount.UtilizationSettings, key)
				}
			}
		}
	return subAccount
}

expose endpoint types consts publicly, so the client (of the client) can provide it's own mapping between type names (if it needs to)

• slack C
• slack U
• custom C
• custom U
• pagerduty C
• pagerduty U
• bigpanda C
• bigpanda U
• datadog C
• datadog U
• victorops C
• victorops U
• R endpoint by id
• R endpoint by name
• list endpoints

https://docs.logz.io/api/#tag/Manage-users

• user CREATE- https://docs.logz.io/api/#operation/createUser
• user UPDATE - https://docs.logz.io/api/#operation/updateUser
• user DELETE - https://docs.logz.io/api/#operation/deleteUser
• user GET - https://docs.logz.io/api/#operation/getUser
• user LIST - https://docs.logz.io/api/#operation/listUsers
• user SUSPEND - https://docs.logz.io/api/#operation/suspendUser
• user UNSUSPEND - https://docs.logz.io/api/#operation/unsuspendUser

Docs https://docs.logz.io/api/#operation/exportSavedObjects

problem

• create_alerts_list test failing

infos

github.com/jonboydell/logzio_client.(*Client).ListAlerts(0xc000041740, 0xc000018011, 0x24, 0x105e1e0, 0xc000041720, 0x851aebb9)
	/Users/jon.boydell/go/src/github.com/jonboydell/logzio_client/client_alerts_list.go:61 +0x12bd

done when

• line 61 GroupByAggregationFields: jsonAlert["groupByAggregationFields"].([]interface{}), can be nil and needs to be dealt with outside the struct initialisation

• get all sub accounts - https://docs.logz.io/api/#operation/getAll
• create a sub account - https://docs.logz.io/api/#operation/createTimeBasedAccount
• get info on sub accounts - https://docs.logz.io/api/#operation/getAllDetailedTimeBasedAccount
• get account info by id - https://docs.logz.io/api/#operation/getDetailedTimeBasedAccount
• get sub account by id - https://docs.logz.io/api/#operation/get
• update sub account - https://docs.logz.io/api/#operation/updateTimeBasedAccount
• delete sub account - https://docs.logz.io/api/#operation/deleteTimeBasedAccount

problem:

• create_alerts_list and create_alerts_get duplicate code for unmarshalling an alert

proposed solution:

• should either use some custom unmarshaller or be refactored into its own function

complete when:

• duplicate code is removed

Go modules has become the canonical way to handle dependencies in a Go project. I would be happy to add support for Go modules in this project.

make sure all deletes are deferred

Some constants are repeated as variables - rename constants so that they're not repeated as variables

users: improve error messaging as returned to client

error messages from logz.io are returned in randomly named error messages with randomly assigned bodies, needs some work....

• need an actual logging strategy
• see json logging from https://www.datadoghq.com/blog/go-logging/ which is more for producing a machine readable log entry

the type of endpoint needs to be part of the struct that holds the endpoint as it's needed for get operations, however - it can't be passed into the struct during create operations - as it's passed in the URI and not the request body...

documentation: update readme with subaccount docs

Endpoint type is case sensitive on read-back but case insensitive on write - this needs to be reflected in equality tests in the client library

• endpoints
• user
• alerts
• client

endpoints: API endpoint "category" to be defined once per namespace

each "type" of API to be managed in it's own package, to prevent name space clashes, to compartmentalise the different APIs and to make testing simpler