Comments (10)
I'll contact the docs examples team to have that rewritten. Thanks for pointing that out.
from aws-sdk-ruby.
Path forward is to do Kernel.warn on various credential sources (EC2 instance, ECS, Process, etc) when they aren't loaded. This should be fine to do because those credential sources are only initialized when certain hint checks are passed (i.e. existence of that relative URI ENV variable).
from aws-sdk-ruby.
Sorry you're running into this - you are right, credentials from ECS TaskRoleArn should just work.
What is AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
set to and are you setting it manually? Are there any other ENV vars set?
from aws-sdk-ruby.
I'm not setting it, it's being set by ECS. It's being set to the following, redacting the UUID as I assume that's something I shouldn't share:
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI=/v2/credentials/[UUID]
There's no other ENV vars that appear auth related. This is a complete set of container ENV vars, minus the ones specific to my app which have a unique, app-specific prefix:
AWS_EXECUTION_ENV=AWS_ECS_EC2
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI=/v2/credentials/[UUID]
ECS_CONTAINER_METADATA_URI=http://169.254.170.2/v3/[UUID]
ECS_CONTAINER_METADATA_URI_V4=http://169.254.170.2/v4/[UUID]
ECS_AGENT_URI=http://169.254.170.2/api/[UUID]
PATH=/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LANG=C.UTF-8
RUBY_VERSION=3.2.3
RUBY_DOWNLOAD_URL=https://cache.ruby-lang.org/pub/ruby/3.2/ruby-3.2.3.tar.xz
RUBY_DOWNLOAD_SHA256=cfb231954b8c241043a538a4c682a1cca0b2016d835fee0b9e4a0be3ceba476b
GEM_HOME=/usr/local/bundle
BUNDLE_SILENCE_ROOT_WARNING=1
BUNDLE_APP_CONFIG=/usr/local/bundle
from aws-sdk-ruby.
From your host, can you try pinging http://169.254.170.2/v2/credentials/[UUID]
to see if this endpoint exists?
We can get more insight if you try initializing these credentials manually:
Aws::ECSCredentials.new(http_debug_output: $stdout) # or your logger
from aws-sdk-ruby.
Also please move away from Resource
models (use Client), they are not recommended because they are hand maintained and do not get updates.
from aws-sdk-ruby.
Thanks for the quick response and good troubleshooting steps. Once I realized how this is supposed to work I was able to root cause the issue: The ECS agent metadata endpoint wasn't responding. Once I resolved that Aws::ECSCredentials.new(http_debug_output: $stdout)
was able to gather credentials as expected. For anyone else here from Google (there were exactly 0 hits for "ecs taskRoleArn Aws::Errors::MissingCredentialsError" last night) the solution was a host reboot.
To try and reproduce the issue I ran a container without network and instantiated a Aws::ECSCredentials
object:
irb(main):002:0> Aws::ECSCredentials.new(http_debug_output: $stdout)
opening connection to 169.254.170.2:80...
opening connection to 169.254.170.2:80...
opening connection to 169.254.170.2:80...
opening connection to 169.254.170.2:80...
opening connection to 169.254.170.2:80...
opening connection to 169.254.170.2:80...
=>
#<Aws::ECSCredentials:0x00007f0b8cbc05b8
@async_refresh=false,
@backoff=#<Proc:0x00007f0b8c26f4a0 /usr/local/bundle/gems/aws-sdk-core-3.191.0/lib/aws-sdk-core/ecs_credentials.rb:177 (lambda)>,
@before_refresh=nil,
@credential_path="/v2/credentials/1ffa0a10-9270-4233-985b-3bdd542f4b88",
@credentials=#<Aws::Credentials access_key_id=nil>,
@expiration=nil,
@host="169.254.170.2",
@http_debug_output=#<IO:<STDOUT>>,
@http_open_timeout=5,
@http_read_timeout=5,
@mutex=#<Thread::Mutex:0x00007f0b8c26f400>,
@port=80,
@retries=5,
@scheme="http">
And I also reproduced the original exception as well. So this isn't a bug, as it failed due to the ecs-agent
not responding, but I do think Aws::ECSCredentials
should throw a warning if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
is set but 169.254.170.2:80 is unresponsive. That's a ENV var I doubt will ever be encountered outside ECS, so if 169.254.170.2 isn't responding that's an error worth bubbling up to the user. If I had realized last night that the http://169.254.170.2
ENV vars did matter I probably would have been able to figure it out.
So I think we can remove the bug
label, but would you be open to leaving this open as a feature request to add a warning level log to the Aws::ECSCredentials class?
from aws-sdk-ruby.
Also please move away from
Resource
models (use Client), they are not recommended because they are hand maintained and do not get updates.
FYI https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/sns-example-send-message.html still documents using Aws::SNS::Resource.
from aws-sdk-ruby.
Related to: #2823
from aws-sdk-ruby.
This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
from aws-sdk-ruby.
Related Issues (20)
- aws-eventstream not available in Ruby 3.2 lambda runtime HOT 6
- presigned_url always adds content-type to signed headers HOT 3
- Aws::EC2::Resources raises `InvalidParameterCombination` when some parameters are specified HOT 2
- aws-sdk-core >= 3.192.0 may be not compatible with AWS SQS APIs in XML format HOT 8
- In Ruby, S3 get_object_attributes() breaks in aws-sdk-core v3.192.0 and newer. HOT 5
- Content Type error in BedrockRuntime client on latest aws-sdk-core version HOT 5
- CloudFront KeyValueStore can't work HOT 3
- stub_responses function in Dynamodb, how can I pass query when stub? HOT 4
- aws-sdk-s3: intermittent SignatureDoesNotMatch errors HOT 7
- get_bucket_policy in 1.52.1 results in Seahorse::Client::NetworkingError Empty or incomplete response body HOT 5
- ECS capacity errors have inconsistent behavior. HOT 2
- Access to the credit_notes? HOT 3
- SImpleDB stopped working after upgrade HOT 9
- Fix INI parsing of whitespace preceding config key HOT 7
- AWS SDK Core 3.192.0 broke older SQS gem ability to receive messages HOT 5
- Breaking Change from Minor Release in `aws-sdk-core` from 3.198.0 to latest HOT 6
- "Passing a single string to Aws::ProcessCredentials.new is insecure" now appearing when using credential_process option in an aws config HOT 4
- Client-side encryption with presigned URLs HOT 11
- Aws::Xml::Parser::ParsingError: xmlParseCharRef: invalid xmlChar value 8 HOT 12
- Getting an intermittent access denied error when running get_object HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aws-sdk-ruby.