Comments (11)
@joshtriplett I actually have a blog post about to be published doing exactly this with real credentials and responds the same way as an ec2 instance (all the SDKs pick it up as native instance profile). I am putting the finishing touches on it and should have it done in the next day or so.
Edit: finished it up and published it showing how to do this
https://medium.com/@slimm609/aws-instance-profile-for-local-development-f144b0a7b8b9
from amazon-ec2-metadata-mock.
@slimm609 That sounds great! Thank you.
from amazon-ec2-metadata-mock.
@slimm609 That's a great use case! Would you be open to adding to our Community Use Cases section with the content? Or would you mind if someone from our team added a section if you don't have bandwidth?
In either case, we would attribute you for the content, but I'm reluctant to simply link the blog post because of the overhead for our team to maintain the integrity of said links.
from amazon-ec2-metadata-mock.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want this issue to never become stale, please ask a maintainer to apply the "stalebot-ignore" label.
from amazon-ec2-metadata-mock.
Still here, still interested.
from amazon-ec2-metadata-mock.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want this issue to never become stale, please ask a maintainer to apply the "stalebot-ignore" label.
from amazon-ec2-metadata-mock.
This issue was closed because it has become stale with no activity.
from amazon-ec2-metadata-mock.
Speaking on behalf of the company I work for, we're very interested in this too. We use ecs-local-container-endpoints and the aws-sdk-v2 (node) works fine when using the IMDS service from the following projects:
- https://github.com/benkehoe/imds-credential-server
- https://github.com/benkehoe/aws-export-credentials
However, aws-sdk-v3 (node) does not seem to work. The default chain never finds a valid credential from the 2 services above.
We need to inject a real temporary credential for the applications running in our containers to work locally and be debuggable. If we cannot get a real/valid credential, we'd need to mock all services using localstack - not an option currently, since it would require seeding those services with tons of data (s3 with files, dynamodb with tables, and so on).
Our security policies do not allow us to pass AWS_ACCESS_KEY_ID
+AWS_SECRET_ACCESS_KEY
to our containers.
This is our docker-compose.override.yml
:
# Original source from: https://aws.amazon.com/blogs/compute/a-guide-to-locally-testing-containers-with-amazon-ecs-local-endpoints-and-docker-compose/
version: "2.1"
networks:
default:
driver: bridge
ipam:
driver: default
credentials_network:
driver: bridge
ipam:
config:
- subnet: "169.254.170.0/24"
gateway: 169.254.170.1
services:
ecs-local-endpoints:
image: amazon/amazon-ecs-local-container-endpoints
volumes:
- /var/run:/var/run
env_file:
- .env
environment:
AWS_EC2_METADATA_SERVICE_ENDPOINT: http://host.docker.internal:8081/
AWS_PROFILE: ${AWS_PROFILE}
networks:
credentials_network:
# This special IP address is recognized by the AWS SDKs and AWS CLI
ipv4_address: "169.254.170.2"
app:
depends_on:
- ecs-local-endpoints
networks:
credentials_network:
ipv4_address: "169.254.170.3"
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
# AWS_REGION: ${AWS_REGION} -- We already load using `env_file` in docker-compose.yml
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI: "/creds"
AWS_EC2_METADATA_SERVICE_ENDPOINT: http://host.docker.internal:8081
ECS_CONTAINER_METADATA_URI: "http://169.254.170.2/v3"
We run either imds-credential-server
or aws-export-credentials
directly on the host, specifying port 8081 and the AWS profile configured in the host AWS cli - the informed profile uses credential_process
along with aws-vault, and that's why we cannot just mount$HOME/.aws
as a volume on the container itself and need a service running on the host to retrieve the credential using the credential process, and expose it via IMDS to the containers. We could install aws-vault
on the containers just for the debugging session, but we try to keep the debug image as similar as possible to the production image (it's a multi-stage Dockerfile).
As I said, it works fine with sdk v2, but not with v3. Since I haven't figured out the reason yet (already invested 6-8 hours), I went after alternatives and arrived here.
from amazon-ec2-metadata-mock.
@jweyrich DId you look at the blog post above? There are several things that are outside the scope of what this tool would do (creating loopback virtual interfaces, templated configs, etc). I created a repo that wraps around ec2-metadata-mock to wrap it all together and works natively on the system with the AWS SDKs. Its in the blog post above as well but here is the repo https://github.com/slimm609/mock-instance-profile
from amazon-ec2-metadata-mock.
@slimm609 I don't think a loopback virtual interface is necessary. See A Guide to Locally Testing Containers with Amazon ECS Local Endpoints and Docker Compose. We define a Docker bridge network in the same CIDR as the IMDS and change the default metadata service endpoint via env var AWS_EC2_METADATA_SERVICE_ENDPOINT
in the application container itself. And it appears that the SDK officially uses its value during the credential chain lookup. The new endpoint has to comply with part of IMDS spec - we tested the 2 services I listed above, and they work with the aws-sdk v2, but not with aws-sdk-v3. I'm trying to figure out the reason, or find an IMDS service that would work with both versions of the aws-sdk. Maybe I'm missing a config to make it work on v3 - already tried these providers: fromNodeProviderChain
, fromContainerMetadata
, fromInstanceMetadata
- from @aws-sdk/credential-providers, following what is mentioned in the upgrading docs.
from amazon-ec2-metadata-mock.
@brycahta would it make sense for the project to support providing valid tokens? I ask because the project name contains "mock" - so it may be far from the initial idea/proposal.
from amazon-ec2-metadata-mock.
Related Issues (20)
- standard_init_linux.go:228: exec user process caused: exec format error HOT 3
- Deploy Helm Chart HOT 3
- Add `tags/instance` path HOT 2
- Emulate specifying `PUT` response hop limit HOT 1
- /latest/user-data ? HOT 2
- Conform the real IMDS by not listing dirs
- Dump all metadata to a file HOT 3
- Userdata formatting HOT 1
- Userdata in getHandlerPairs()? HOT 1
- marketplaceProductCodes field is typed as a string instead of a list of strings HOT 1
- Passing null value in AEMM config parameter HOT 6
- Optionally support the removal of instances tags within returned metadata HOT 2
- [bug] cannot add a new tag using config overrides HOT 3
- Provide this mock as a Java library HOT 2
- [feature request] Watch config file for changes and reload HOT 2
- Programmatic use in Go tests HOT 2
- Incorrect Content-Type response header HOT 4
- IAM credentials should have realistic values HOT 2
- Missing EC2ws header HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from amazon-ec2-metadata-mock.