cloudtools / awacs Goto Github PK
View Code? Open in Web Editor NEWPython library for AWS Access Policy Language creation
License: BSD 2-Clause "Simplified" License
Python library for AWS Access Policy Language creation
License: BSD 2-Clause "Simplified" License
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/permissions-reference-cw.html
We are missing CloudWatch dashboard actions.
I was going recreating this AWS policy example with Troposphere and awacs and I found that awacs was missing a couple actions.
codecommit:CancelUploadArchive
codecommit:GetCommit
codecommit:GetUploadArchiveStatus
codecommit:UploadArchive
cloudformation:DeleteChangeSet
More may be missing, I didn't look at the spec, these actions just weren't implemented and I was trying to use them.
Hi,
Currently it seems like awacs.aws.Statement
's Resource attr can only be a list
(https://github.com/cloudtools/awacs/blob/master/awacs/aws.py#L143)
It would be useful to allow passing in a troposphere.Ref
object because then you can have a CommaDelimitedList parameter which contains the resources you want to create the policy for.
I guess this could be done with a callable, like the Effect
type check in Statement
, although it feels like there could be a more general solution that could be useful for all types, not just Statement.
Thanks
Hi. The Statement class only allows a single principal as argument. When a list of principals has the same statement, I have to repeat the statement for each principal instead of creating a single statement with a list of principals.
Would be cleaner to pass a list of principals and have a single statement for a list of principals. Thanks!
My sample code is below. When I run it, which is straight out of the examples, I get the following error:
Resource=[s3.ARN("*"), ],
File "C:\Python27\lib\site-packages\awacs\__init__.py", line 111, in __init__
sup.__init__(None, props=self.props, **kwargs)
File "C:\Python27\lib\site-packages\awacs\__init__.py", line 40, in __init__
self.__setattr__(k, v)
File "C:\Python27\lib\site-packages\awacs\__init__.py", line 84, in __setattr__
(self.type, name))
File "C:\Python27\lib\site-packages\awacs\__init__.py", line 46, in __getattr__
raise AttributeError(name)
AttributeError: type
cfnrole = awsinstance.add_resource(Role(
"CFNRole",
AssumeRolePolicyDocument=Policy(
Statement=[
Statement(
Effect=Allow,
Action=[AssumeRole],
Principal=Principal("Service", ["ec2.amazonaws.com"])
)
]
),
Path='/',
Policies=[
Policy(
PolicyName="S3AccessForInstances",
PolicyDocument=Policy(
Statement=[
Statement(
Action=[s3.ListAllMyBuckets, s3.GetBucketLocation],
Effect=Allow,
Resource=[s3.ARN("*"), ],
),
],
)
),
]
))
The _condition_strings
has some wrong condition name value:
StringEqualsIgnoresCase
should be StringEqualsIgnoreCase
(without the s in Ignore)
in addition the condition StringNotEqualsIgnoreCase
is missing.
awacs.s3.ARN
sets account to the empty string.
Lines 18 to 21 in aba4018
Actions for CodeArtifact need to be added and also the GetServiceBearerToken for STS:
We're using KMS asymmetric keys to sign and we'd like to add permissions programmatically. It appears the current 0.9.8
version was released in February and KMS was updated to include Sign
a week ago. Can you please cut a new release so we can incorporate this?
Thanks!
when installing awacs using pip I get
Could not find .egg-info directory in install record for awacs==0.4.2
This triggers a bug in pip with removing an index file:
https://mail.python.org/pipermail/distutils-sig/2013-August/022528.html
I'll try and submit an issue on pypa/pip, but it would potentially be nice to remove the cause of the warning.
I've noticed in troposphere that with self.propnames = props.keys()
the object can no longer be pickled.
Does AWSObject
need to be able to be pickled?
Is it even necessary to cache the keys? name in self.props
(.keys()
is not necessary) should be just as fast as name in self.propnames
.
I don't see a reason for caching here.
See b4e4008#diff-7026bda438cf5fffa3496ee838ea9504d1962b37c3e1c573f34b09ebe4b4588cL21-R21
I am trying to make a awacs.aws.Statement
that will generate the following template:
Parameters:
Parameter-SomeArnList:
Description: Comma-delimited list of some ARNs
Type: CommaDelimitedList
Resources:
SomePolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
PolicyDocument:
Version: '2012-10-17'
Statement:
- Action:
- cloudformation:DescribeStacks
Effect: Allow
Resource: !Ref 'ParameterSomeArnList'
where Parameter-SomeArnList
is a CommaDelimitedList parameter.
Using troposphere and awacs, I construct this template as follows:
import troposphere
from troposphere import iam
import awacs
ex_template = troposphere.Template()
ex_template.add_parameter(
troposphere.Parameter(
title="ParameterSomeArnList",
Type="CommaDelimitedList",
Description="Comma-delimited list of some ARNs"
)
)
ex_template.add_resource(
iam.ManagedPolicy(
title="Example",
PolicyDocument=awacs.aws.Policy(
Version="2012-10-17",
Statement=[
awacs.aws.Statement(
Action=[awacs.aws.Action(prefix="cloudformation", action="DescribeStacks")],
Effect="Allow",
Resource=troposphere.Ref("ParameterWorkerNodeCfnArns")
)
]
)
)
)
ex_template.to_yaml()
you will hit an error though:
Traceback (most recent call last):
File "<stdin>", line 10, in <module>
File "/lib/python3.7/site-packages/awacs/__init__.py", line 128, in __init__
sup.__init__(None, props=self.props, **kwargs)
File "/lib/python3.7/site-packages/awacs/__init__.py", line 39, in __init__
self.__setattr__(k, v)
File "/lib/python3.7/site-packages/awacs/__init__.py", line 80, in __setattr__
self._raise_type(name, value, expected_type)
File "/lib/python3.7/site-packages/awacs/__init__.py", line 89, in _raise_type
(name, type(value), expected_type))
TypeError: Resource is <class 'troposphere.Ref'>, expected <class 'list'>
You can make a very minor change to awacs/aws.py
to fix this issue, although it requires awacs to import troposphere.
This is a tracking issue for awacs 2.0 which will remove Python 2 support and be Python 3 only.
IAM Policy Generator supports "Amazon Cognito User Pools", having these actions:
(source: IAM Policy Generator JS source code.)
We need support for this in awacs.
Gov Cloud and China regions use an alternate partition name for ARNs. The following docs and code from aws which show how to properly build a region-agnostic arn: aws arn docs
TL;DR
example arn
arn:partition:service:region:account-id:resourcetype:resource
partition : The partition that the resource is in. For standard AWS regions, the partition is aws. If you have resources in other partitions, the partition is aws-partitionname. For example, the partition for resources in the China (Beijing) region is aws-cn.
awscli
TL;DR
def _get_policy_arn_suffix(region):
region_string = region.lower()
if region_string.startswith("cn-"):
return "aws-cn"
elif region_string.startswith("us-gov"):
return "aws-us-gov"
else:
return "aws"
Hello,
I am creating an S3 policy for my lambda function, which requires the ListObjectVersions. I noticed that the current version doesn't support or is not listed as one of the permissions. Could you please add these permissions.
Thanks,
/santosh
Any chance to give some luv to current PRs?
Per https://docs.aws.amazon.com/apigateway/latest/api/API_Operations.html, ApiGateway has a TagResource action. Using this call, or apparently any action that adds tags to a resource like CreateDomainName, requires the permission apigateway:TagResource, but this is missing as an action in awacs/apigateway.py
The autoscaling.py file is missing the following lines:
AttachLoadBalancers = Action(prefix, 'AttachLoadBalancers'),
DetachLoadBalancers = Action(prefix, 'DetachLoadBalancers')
These were actually missing from the AWS console until recently due to an oversight on their end. They have always been valid permissions, but just didn't exist in the "IAM Role Generator" UI. I filed a support request with AWS and they added them.
When serializing to JSON, Condition of the aws module references a condition instance variable of ConditionElement
def JSONrepr(self):
d = {}
for c in self.conditions:
d[c.condition] = c.get_dict()
However, ConditionElement has instance variables of key, value, and cond_dict. This situation causes an issue when serializing
...
File "/projects/aws/python-3.4/lib64/python3.4/site-packages/awacs/aws.py", line 106, in JSONrepr
d[c.condition] = c.get_dict()
AttributeError: 'ConditionElement' object has no attribute 'condition'
The Condition class utilizes a list of ConditionElement instances. These instances could be serialized provided the ConditionElement also provides serialization.
From AWS Support's documentation:
AWS Support does not let you allow or deny access to individual actions;
therefore your policy must use the "Action": "support:*" to use the
AWS Support Center or to use the AWS Support API.
Therefore the actions in support.py seem useless if not misleading. And there's no predefined action in awacs that covers support:*
.
Can the actions be used for anything else than IAM Policies? Am I missing something?
When this code (that is found in the README) is run...
from awacs.iam import ARN as IAM_ARN
from awacs.s3 import ARN as S3_ARN
account = "123456789012"
user = "user/Bob"
pd = Policy(
Version="2012-10-17",
Id="S3-Account-Permissions",
Statement=[
Statement(
Sid="1",
Effect=Allow,
Principal=Principal("AWS", [IAM_ARN(account, user)]),
Action=[Action("s3", "*")],
Resource=[S3_ARN("my_corporate_bucket/*"),],
),
],
)
print(pd.to_json())
It does not produce the json policy as specified in the README. The IAM_ARN is wrong.
Expected: "arn:aws:iam:123456789012:user/Bob:"
Actual: "arn:aws:iam:user/Bob::123456789012"
Would it be possible to release a new version? I could use the addition of airflow added in Update 2020-50.
This is related to #84
S3 bucket ARN cannot contain region information, and the current implementation to generate the ARN partition is based on the region which should not be specified.
This is an issue for buckets in China region (e.g. cn-north-1).
This is an example of a valid China bucket: arn:aws-cn:s3:::my_china_bucket
This affects any region-less services like S3 and IAM in China accounts.
aws_partition should be at least optional.
def __init__(self, service, resource, region='', account='', aws_partition=''):
region_string = region.lower()
if not aws_partition:
if region_string.startswith("cn-"):
aws_partition = "aws-cn"
elif region_string.startswith("us-gov"):
aws_partition = "aws-us-gov"
else:
aws_partition = "aws"
self.data = "arn:%s:%s:%s:%s:%s" % (
aws_partition, service, region, account, resource)
awacs.aws.Policy(Version='2012-10-17', Statement=[]) # No error
awacs.aws.Policy(Version='2012-10-17', statement=[]) # Try to raise on line 84, but .type is never set, so we raise on line 46
GitHub's storage layer is down but here is a patch that I presume does the right thing.
--- a/awacs/__init__.py 2015-10-18 15:35:38.000000000 -0700
+++ b/awacs/__init__.py 2015-10-18 15:40:23.000000000 -0700
@@ -20,6 +20,10 @@
self.props = props
# Cache the keys for validity checks
self.propnames = props.keys()
+ if type is None:
+ self.type = type(self).__module__ + '.' + type(self).__name__
+ else:
+ self.type = type
# unset/None is also legal
if name and not valid_names.match(name):
@markpeek could please create a new release? Thank you :)
You might already be familiar with https://github.com/iann0036/iam-dataset, but it's a rather handy reference that packages up a ton of useful information about AWS IAM and is used downstream in a number of tools. You might consider ripping out the existing scraping tooling in this repository - if you don't want to maintain it - and instead sourcing the package's data from iam-dataset!
This is per Chris Shenton on the cloudtools-dev group:
Been using tropo for a few weeks and started using AWACS so I can more easily create IAM Roles and Policy documents. Liking it a lot, thanks, guys.
But I'm stuck trying to reference a resource in an AWACS policy, I can't figure how to build the ARN. The docs show hard-wired names, e.g.:
S3: Resource=[s3.S3_ARN("myBucket")],
SQS: Resource=[sqs.SQS_ARN(region, account, "queue1"), ],
For things that need Region and Account, is there a way to get those without hard-wiring them in my tropo script? I was hoping to use Ref("AWS::Region") and Ref("AWS::AccountID") but those generate resources like:
"Resource": [
"arn:aws:dynamodb:<troposphere.Ref object at 0x10978c128>:<troposphere.Ref object at 0x10978c1d0>:table/JobStateDB"
],
I'd also like to avoid hard-wiring my resource name; for example, I have a DynamoDB defined like:
r_jobstate_db = t.add_resource(
dynamodb.Table(
"JobStateDB",
AttributeDefinitions=[dynamodb.AttributeDefinition("job", "S")],
KeySchema=[dynamodb.Key("job", "HASH")],
ProvisionedThroughput=dynamodb.ProvisionedThroughput(
Ref(p_jobstate_readunits),
Ref(p_jobstate_writeunits),
)
))
And I want to reference it in AWACS. Below, I know the Ref()s fail as above; I think I'm relying on knowing the implementation when I access the ".title" attribute:
Statement(
Action=[awacs.dynamodb.PutItem, ],
Effect=Allow,
Resource=[
awacs.dynamodb.ARN(
Ref("AWS::Region"),
Ref("AWS::AccountID"),
r_jobstate_db.title,
),
],
Any guidance on how I can:
* Get AWS "Region" and "AccountID" to build the ARN without hard-wiring
* Get the resources name to build the ARN without peeking at code
Or is there a totally smarter way to do this, like:
Resource=[awacs.dyamodb.MagickARN(r_jobstate_db)]
Thanks!
This seems like something we should investigate. Not sure if this is:
Anyway, I think it's curious enough that we should have a bug for it just to make sure it doesn't get lost.
Can the following new policies be added in? They were released recently.
glue:GetDataCatalogEncryptionSettings
glue:PutDataCatalogEncryptionSettings
glue:CreateSecurityConfiguration
glue:GetSecurityConfiguration
glue:GetSecurityConfigurations
glue:DeleteSecurityConfiguration
It appears that the policies.js bundle that's being parsed to generate the .py files is missing some dynamo actions. Specifically dynamodb:DescribeTimeToLive and dynamodb:UpdateTimeToLive.
This page appears to be missing UpdateTimeToLive
https://docs.aws.amazon.com/IAM/latest/UserGuide/list_dynamodb.html
This page maybe has the most correct set of references?
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/api-permissions-reference.html
It might make sense for the gen.py
process to reconcile the policies.js source against some of the html documentation. I'd be willing to take a crack at that with a pull request if you're interested.
awacs/sqs.py lists ChangeMessageVisibilityBatch as a permission but if I try and use it in a queue policy manually I get:
Failed to save changes to the policy document. Reason: com.amazonaws.AmazonServiceException: Value SQS:ChangeMessageVisibilityBatch for parameter ActionName is invalid. Reason: Please refer to the appropriate WSDL for a list of valid actions. (Service: AmazonSQS; Status Code: 400; Error Code: InvalidParameterValue;
The WSDL at http://queue.amazonaws.com/doc/2012-11-05/QueueService.wsdl doesn't seemed to clarify (as ChangeMessageVisibilityBatch is listed) but I don't fully understand it. But it seems like ChangeMessageVisibilityBatch isn't valid, and it is not listed on http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/UsingIAM.html.
Hi,
I don't suppose you could release a new version? There are quite a few new services only on master, like SWF.
Cheers
Craig
I think the code for the ConditionElements in awacs/aws.py
should be generated instead of manipulating globals.
The current implementation makes it impossible for static code checkers (linters, type checkers, โฆ) to handle it correctly. Eg. pylint reports a no-name-in-module
error for from awacs.aws import StringEquals
.
Hi,
at least in the lates t(0.6.0) version the IAM ARN function returns something else than in `0.5.4``
. This is no more a valid ARN, therefore I consider this function broken in the new version.
import awacs.iam
foo_arn = awacs.iam.ARN('123456789012', 'root')
print foo_arn.data
this is an valid ARN
$ python test.py
arn:aws:iam::123456789012:root
this ARN is invalid
$ python test.py
arn:aws:iam:root::123456789012
Hello,
There were some upates added for a few missing permissions that I opened an issue for about a month back. I'm hoping we can get a release with these changes?
Thanks!
Here is the error:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/awacs/__init__.py", line 44, in __getattr__
return self.properties.__getitem__(name)
KeyError: 'type'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "temp.py", line 16, in <module>
print(PolicyDocument.to_json())
File "/usr/local/lib/python3.5/dist-packages/awacs/aws.py", line 174, in to_json
return json.dumps(p, cls=awsencode, indent=indent, sort_keys=sort_keys)
File "/usr/lib/python3.5/json/__init__.py", line 237, in dumps
**kw).encode(obj)
File "/usr/lib/python3.5/json/encoder.py", line 200, in encode
chunks = list(chunks)
File "/usr/lib/python3.5/json/encoder.py", line 429, in _iterencode
yield from _iterencode_dict(o, _current_indent_level)
File "/usr/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict
yield from chunks
File "/usr/lib/python3.5/json/encoder.py", line 324, in _iterencode_list
yield from chunks
File "/usr/lib/python3.5/json/encoder.py", line 436, in _iterencode
o = _default(o)
File "/usr/local/lib/python3.5/dist-packages/awacs/__init__.py", line 127, in default
return obj.JSONrepr()
File "/usr/local/lib/python3.5/dist-packages/awacs/__init__.py", line 99, in JSONrepr
(k, self.type))
File "/usr/local/lib/python3.5/dist-packages/awacs/__init__.py", line 46, in __getattr__
raise AttributeError(name)
AttributeError: type
here is the sample code generating that:
from awacs.aws import Action, Policy, Statement
policy=Policy( Statement=[ Statement() ]) print(policy.to_json())
adding Effect="Allow"
fixes it:
from awacs.aws import Action, Policy, Statement
policy=Policy( Statement=[ Statement(Effect="Allow") ]) print(policy.to_json())
It seems with the latest release, it is no longer possible to specify CanonicalUser
as a principal? This type of principal is used for S3 BucketPolicy, with Statement:
aws.Statement(
Effect=aws.Allow,
Principal=aws.Principal(
"CanonicalUser", "some canonical id"
),
Action=[
aws.Action("s3", "GetObject"),
],
Resource=[
Join("", ["arn:aws:s3:::", Ref(self.bucket_name), "/*"]),
],
),
the above now fails with:
ValueError: Principal must be one of: AWS, Federated, Service
StringEquals is referenced many times through the code but the function is no longer used.
as seen in these examples:
Line 7 in 7ff6195
and the description of how to use ConditionalElement here
Line 90 in 7ff6195
How is this functions supposed to work? I saw it referenced in the _conditional_strings variable here:
Line 215 in 7ff6195
however I am not sure how i am supposed to use the following code to create the condition as it returns a NoneType Object when using it like so: make_condition('StringEquals', 'StringEquals')
Line 229 in 7ff6195
the use of this conditional implementation is unclear or broken.
Does this library support a deserialization method; in other words, the reverse of to_json()? If I have JSON/Dict repr of an AWS object, can I convert it back to an object?
When creating policies for cloudfront, resource level specifications are not allowed and example. So when creating policies the creator has to remember to change
"Resource": [
"arn:aws:cloudfront:::*"
]
to
"Resource": [
"*"
]
for the policy to be correct.
As documented in https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_multi-value-conditions.html - should be possible to make a condition like:
"Condition": {
"ForAllValues:StringLike": {
"dynamodb:requestedAttributes": [
"PostDateTime",
"Message",
"Tags"
]
}
}
Seem valuable to anyone else to add?
I am trying to create an S3 bucket policy like this
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicAccess",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/*"
}
]
}
But the way Principal
is written now, it only supports the format
"Principal": {
"Federated": "accounts.google.com"
}
:-(
I think this were added fairly recently with tags support for AWS Lambda.
/cc @phobologic
I spent about 3h to find out that the policy was invalid because it is not 'service', but 'Service'.
Is there a way to prevent watch for those typos?
The error shows up only at create_stack or stack_updates.
Surprisingly, the template validation is successful.
AWS has a relatively new service prefix (eks-auth
) that exposes a single action. It'd be great to have this in awacs
!
I was trying to create an IAM role document using troposphere and awacs but ran into an issue with using the Policy
and Statement
classes for the AssumeRolePolicyDocument
attribute of a Role.
The Statement
class requires that the Action
and Principal
be a list and not single items. While this is true for regular IAM policies, it is not true for the Trust Relationship documents required for IAM Instance Roles.
For those policy documents - the Action and Principal must be a single item and not a list.
Here is some code that works with awacs/troposphere to generate a Policy.
iam_role = iam.Role(
name='IAMRole',
AssumeRolePolicyDocument=awacs_aws.Policy(
Statement=[
awacs.aws.Statement(
Effect=awacs.aws.Allow,
Principal=[awacs.aws.Principal(principal="Service", resources=["ec2.amazonaws.com"])],
Action=[awacs.sts.AssumeRole]
)
]
),
Path="/",
Policies=
...
)
The policy document generated by the code is invalid according to AWS and causes a syntax error in the UI when you try to save it.
{
"Statement": [
{
"Action": [
"sts:AssumeRole"
],
"Effect": "Allow",
"Principal": [
{
"Service": [
"ec2.amazonaws.com"
]
}
]
}
]
}
In order to make the above document valid, you just need to make both Principal and Action a single item instead of a list, and then it saves correctly in the UI.
I am not sure the best way to fix this as the Statement class is shared, as is the Policy class. Or if its even worth fixing. I just wanted to share my workaround incase anyone else hit it.
I just used a regular dict for this policy like:
name='IAMRole',
AssumeRolePolicyDocument={
"Statement": [
{
"Action": awacs.sts.AssumeRole,
"Effect": awacs.aws.Allow,
"Principal": awacs.aws.Principal(principal="Service", resources=["ec2.amazonaws.com"])
}
]
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.