mergermarket / acuris-aws-es-connection Goto Github PK

Hi,

When I'm compiling the code with this library I get:

node_modules/@acuris/aws-es-connection/dist/index.d.ts:1:8 - error TS1192: Module '"/Users/..../node_modules/aws-sdk/index"' has no default export.

1 import AWS from 'aws-sdk';
         ~~~

I'm pretty new to typescript so I could be mistaken ... but when I change following line

into

import * as AWS from 'aws-sdk';

I don't have the error anymore ....

Am I doing something wrong in my project or is it something to improve here?

Tried updating from the old Node client and came across this package. I am able to connect locally, but once my Lambda is deployed to AWS the connection is refused. I have ensured that the Lambda's execution IAM role has the proper access.

I was just curious if you'd tested this within a Lambda before I dug too much deeper.

I had some working code that could access my AWS elasticsearch domain to write to an index and search it. Now I'm trying to add authentication and using this package my code no longer works. I get a rather vague "Response Error. Code 403" Have I missed something obvious?

const credentials = new Credentials(MY_ACCESS_KEY_ID, MY_SECRET_KEY);
  const AWSConnection = createAWSConnection(credentials);
  client = awsCredsifyAll(
    new Client({
      Connection: AWSConnection,
      node: 'https://MY_AWS_NODE.us-east-2.es.amazonaws.com/'
    }));

  client.ping({}, { requestTimeout: 20000 }, (err: any, response: any) => {
    if (err) {
      console.log(err);
    } else {

      beginSetupIndexes().catch(console.log);
      // performSearch().catch(console.error);
    }
  });

I get the error 403 in the ping() callback

{ ResponseError: Response Error
    at IncomingMessage.response.on (node_modules\@elastic\elasticsearch\lib\Transport.js:240:25)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1064:12)
    at _combinedTickCallback (internal/process/next_tick.js:139:11)
    at process._tickDomainCallback (internal/process/next_tick.js:219:9)
  name: 'ResponseError',
  meta: 
   { body: true,
     statusCode: 403,
     headers: 
      { date: 'Wed, 02 Sep 2020 10:36:37 GMT',
        'content-type': 'application/json; charset=UTF-8',
        'content-length': '415',
        connection: 'keep-alive',
        'access-control-allow-origin': '*' },
     warnings: null,
     meta: 
      { context: null,
        request: [Object],
        name: 'elasticsearch-js',
        connection: [Object],
        attempts: 0,
        aborted: false } } }
Debugger attached.
Waiting for the debugger to disconnect...

Process finished with exit code 0

I suspect this module is not working properly when using a Custom Endpoint with a CNAME / A in regions other than 'us-east-1'.

Wen configuring the Route53, I tried both CNAME record to the VPC endpoint and A record to the respective IP. When trying to open a connection, this error is logged:

Credential should be scoped to a valid region, not 'us-east-1'. Credential should be scoped to correct service: 'es'. 

The error can be replicated by creating a Custom Endpoint in AWS ElasticSearch in a region other than 'us-east-1', having with a proper ACM certificate for the Custom Endpoint and try to connect to it.

Using the regular VPC endpoint works just fine.

Do not forget to set the AWS SDK region in first place:

  AWS.config.update(new Config({ region: 'some-region' }));

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch @acuris/[email protected] for the project I'm working on.

I noticed that I couldn't pass a simple Credentials object to sign() because it was importing a much more complex one from aws-sdk than what sign() from aws4 actually needs

Here is the diff that solved my problem:

diff --git a/node_modules/@acuris/aws-es-connection/dist/index.d.ts b/node_modules/@acuris/aws-es-connection/dist/index.d.ts
index a3a5cf2..e83b45f 100644
--- a/node_modules/@acuris/aws-es-connection/dist/index.d.ts
+++ b/node_modules/@acuris/aws-es-connection/dist/index.d.ts
@@ -1,4 +1,4 @@
-import { Credentials } from 'aws-sdk/global';
+import { Credentials } from 'aws4';
 import { ClientOptions } from '@elastic/elasticsearch';
 export declare function createAWSConnection(awsCredentials: Credentials): ClientOptions;
 export declare const awsGetCredentials: () => Promise<Credentials>;

This issue body was partially generated by patch-package.

I am using the latest version of

• "@acuris/aws-es-connection": "^1.0.1"
• "@elastic/elasticsearch": "^7.5.0"

here is my sample code

import { awsCredsifyAll, createAWSConnection } from '@acuris/aws-es-connection';
import { Client } from '@elastic/elasticsearch';

const awsCredentials = new aws.EnvironmentCredentials('AWS');

  const elasticsearchClient = awsCredsifyAll(
    new Client({
      node: `https://${CONTENT_FINDER_ES}/`,
      Connection: createAWSConnection(awsCredentials),
      requestTimeout: 600 * 1000,
    }),
  );

I am getting below mentioned TS Lint error,

Type 'Client' is missing the following properties from type 'Client': count_percolate, countPercolate, delete_template, deleteTemplate, and 9 more.ts(2345)

Can somebody help?

here is the full error

Amazon did a great thing of releasing a set of tiny modules instead of a dinosaur they had before.

I'm planning to send PR, which removes peerDependency / devDependency of aws-sdk and add @aws-sdk/client-elasticsearch-service

Using the same code given in the document throws below error

TypeError: Class extends value undefined is not a constructor or null
at generateAWSConnectionClass (/var/task/node_modules/@acuris/aws-es-connection/dist/index.js:10:56)
at createAWSConnection (/var/task/node_modules/@acuris/aws-es-connection/dist/index.js:51:21)
at getESClient (/var/task/index.js:20:25)
at async exports.handler (/var/task/index.js:38:27)

my dependecies

"dependencies": {
    "@acuris/aws-es-connection": "^2.3.0",
    "@elastic/elasticsearch": "^8.11.0"
  }

const awsCredentials = await awsGetCredentials(); returns the credentials successfully but it fails in const AWSConnection = createAWSConnection(awsCredentials);

can you help resolve this issue?

I have a typescript "client" class where I connect to elasticsearch inside the constructor.
now I have to switch to connect to AWS ES, I came across your package.
but I can't call the async awsGetCredentials() method inside a (typescript) constructor.
is there a sync version?

Thanks to corporate battles, AWS has forked the Elasticsearch client to the new https://github.com/opensearch-project/opensearch-js client. Can we get this working with that? Or should I fork it?

First, thank you for writing this package! It's been working great for a project I'm working on. However, I was wondering why package.json caps the @elastic/elasticsearch dependency at 7.9.1. I saw that it wasn't working with 7.9.1 briefly, but it looks like you resolved that. We're using 7.11.0 and it works just fine (even with the unit test). We're happy to submit a PR but it looks like a quick change to your end.

Hi,
Thanks for making this package. Till now we were using http-aws-es and it was all fine. We decided to upgrade es and native es client package so we thought of using this package. We have two use cases

• Calling it from Lambda
• Calling it from EC2

Code below works fine from AWS lambda but gives error while using it from EC2 box.

const {Client} = require("@elastic/elasticsearch");
const {createAWSConnection, awsCredsifyAll} = require("@acuris/aws-es-connection");

const AWSConnection = createAWSConnection(AWS.config.credentials);

const elasticsearchConfig = {
  "node": "elastic-url"
  "Connection": AWSConnection
};

const nativeClient = awsCredsifyAll(new Client(elasticsearchConfig));

nativeClient.search({index: "someindex", "body": {}}).then(res => console.log(res));

This is the error we recieve

TypeError: Cannot read property 'getPromise' of null
    at Client.search (/tmp/test/node_modules/@acuris/aws-es-connection/dist/index.js:37:18)

Please note that EC2 has correct set of permissions so no issue around that. Looks like AWS credentials are coming as null.

Can you please help here?

After upgrading the peer @elastic/elasticsearch dependancy to 7.9.1 - after a period of time I get the following error:

the security token included in the request is expired