timothymugayi / boto3-sqs-extended-client-lib Goto Github PK

Expected Behavior

PyPi is up to date with Github code.

Current Behavior

PyPi code is 3 years out of date.

Steps to Reproduce

1. Run pip install pysqs-extended-client
2. Check files against the github files
3. The files do not include the FIFO code.

Expected Behavior

Functionality to be able to read a message from a different account to the one that publishes the message.

Current Behavior

ACL is always set to "private" when uploading the file to S3, so it is not possible to read the message from a different account.
As a work-around I tried assuming a role in the other account, but it seems the credentials passed to the SQS client are ignored, in favor of the environment variables set in the config.py file.

The message format produced by "boto3-sqs-extended-client-lib" and "amazon-sqs-java-extended-client-lib" are different if the message size is greater than 256 KB.

boto3-sqs-extended-client-lib (JSON object)

e.g.
{"s3BucketName":"some-bucket","s3Key":"some-uuid"}

amazon-sqs-java-extended-client-lib (JSON array)

e.g.
["com.amazon.sqs.javamessaging.MessageS3Pointer",{"s3BucketName":"some-bucket","s3Key":"some-uuid"}]

The purpose of raising this issue is to align the message format with Amazon SQS Extended Client Library for Java mentioned in the official documentation.

References:
https://docs.amazonaws.cn/en_us/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-s3-messages.html
https://github.com/awslabs/amazon-sqs-java-extended-client-lib

Expected Behavior

I want send_message to accept message group ID and deduplication ID as a parameters so that FIFO queues can be pushed to from this utility.

Current Behavior

Currently the client has no support for message group and deduplication IDs, so it only supports standard queues.

What is the current behavior?

Failure Information (for bugs)

Pushing to a FIFO queue from this client raises the following botocore error:

botocore.exceptions.ClientError: An error occurred (MissingParameter) when calling the SendMessage operation: The request must contain the parameter MessageGroupId.

Steps to Reproduce

Please provide detailed steps for reproducing the issue.

1. Create a .fifo queue
2. Attempt to push to the queue via SQSClientExtended.send_message

Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

• Firmware Version: 1.14.0
• Operating System: Ubuntu 19.04
• package version: Latest

Expected Behavior

I would like to send multiple sqs messages

Current Behavior

I am able to send one sqs message but not more than that.

Failure Information (for bugs)

RESERVED_ATTRIBUTE_NAME.value

sqs.send_message(AWS_SQS_QUEUE_URL,message[0])
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python3.6/dist-packages/pysqs_extended_client/SQSClientExtended.py", line 208, in send_message
raise ValueError("Message attribute name {} is reserved for use by SQS extended client.".format(SQSExtendedClientConstants.RESERVED_ATTRIBUTE_NAME.value))
ValueError: Message attribute name SQSLargePayloadSize is reserved for use by SQS extended client.

Steps to Reproduce

Please provide detailed steps for reproducing the issue.

1. from pysqs_extended_client.SQSClientExtended import SQSClientExtended
2. sqs = SQSClientExtended(None,None,None, loc)
3. sqs.send_message(AWS_SQS_QUEUE_URL,message[0])

Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

• Firmware Version:
• Operating System:
• package version:

Failure Logs

Please include any relevant log snippets or files here.

Expected Behavior

Poetry should be able to install this lib.

Current Behavior

but it gives error:

Because moto (1.3.13) depends on botocore (>=1.12.86)                                                                                                                                                           
   and pysqs-extended-client (0.0.1) depends on botocore (1.10.80), moto (1.3.13) is incompatible with pysqs-extended-client (0.0.1).                                                                             
  And because pysqs-extended-client (0.0.1) depends on moto (1.3.13), pysqs-extended-client is forbidden. 

Failure Information (for bugs)

see above

Steps to Reproduce

1. install poetry
2. add pysqs-extended-client = "*" to [tool.poetry.dependencies]
3. run poetry install

Context

python 3.8
mac intel chip

My guess

pip install can install it successfully probably it ignore dependency moto which is not used actually?
If moto is not really used, removing it can solve the problem.

For some reason, the following three lines cause the client to crash with the message Message attribute name SQSLargePayloadSize is reserved for use by SQS extended client. sometimes:

Workaround: when removing those lines, I see no more error and everything works as expected every time.

The way I call the send_message method is using the following signature without adding any message attribute:
sent_message = sqs.send_message(url, message)

ie. message_attributes should be equal to {} and should not contain the so-called RESERVED_ATTRIBUTE_NAME right? I did not investigate why the above ValueError would be raised from time to time. Just leaving an issue here if anyone wants to go further or encounters the same problem as the workaround is pretty simply to remove this check and exception raise.