awspilot / dynamodb-oop Goto Github PK
View Code? Open in Web Editor NEWSpeak fluent DynamoDB, write code with fashion, I Promise() ๐
Home Page: https://awspilot.dev
License: MIT License
Speak fluent DynamoDB, write code with fashion, I Promise() ๐
Home Page: https://awspilot.dev
License: MIT License
Hi
When I try to perform the query by using .select(DynamoDB.ALL).
It raises exception
{ [InvalidParameterType: Expected params.AttributesToGet[0] to be a string]
message: 'Expected params.AttributesToGet[0] to be a string',
code: 'InvalidParameterType',
time: Tue Jun 07 2016 23:30:16 GMT-0700 (PDT) }
Hi,
I am my config.json contains region. I am using
// Load the AWS SDK for Node.js
var AWS = require('aws-sdk');
// Load credentials and set region from JSON file
AWS.config.loadFromPath('config/config.json');
//AWS.config.update({region: 'us-east-1'});
var $db = new AWS.DynamoDB();
var DynamoDB = require('@awspilot/dynamodb')($db);
But I still get
ConfigError: Missing region in config error .
Config.json looks like
{ "accessKeyId": "XXXXXXXXXX",
"secretAccessKey": "##############",
"region": "us-east-1"
}
Trying to get an item, it's missing the string set field.
It seems that only "AND" is supported right now. Is it the case?
Hi
I am trying to use the batch_insert query using the lip but it doesn't seem to be working.
-Randeep
If I use the boolean property "false", the property is presented as "undefined" in the database.
What I have done wrong, or is this a bug?
Thanks :-)
Hi I believe its part of a the recent update, of all my dynamodB requests, I see one last object to be completely empty.
It's happening with the scan operation.
DynamoDB.table('TableName')
.scan (function (err, data) {
if (err) {
cb(err, null);
}
cb(null, data);
});
I use Lambda Layer arn:aws:lambda:ap-northeast-1:452980636694:layer:awspilot-dynamodb-2_0_0-beta:1
.
I have problem while transaction on the layer.
Example followings:
const DynamoDB = require('@awspilot/dynamodb');
exports.handler = async (event) => {
const db = DynamoDB({});
db.schema({
TableName: "my-table",
KeySchema: [{ KeyType: 'HASH', AttributeName: 'id' }],
});
const tran = db.transact();
const table = tran.table("my-table");
console.log(table);
console.log(table.insert);
const response = {
statusCode: 200,
body: JSON.stringify('Hello from Lambda!'),
};
return response;
};
The outputs were followings:
INFO Transact {
pending: { tableName: 'my-table' },
TransactItems: [],
// Omited....
}
INFO undefined
insert
method undefineded.
I have unittest code on local.
The local environment using npm package.
I added logging to unittest code samely.
The local environment outputs were folliowings:
console.log
Transact {
pending: { tableName: 'my-table' },
TransactItems: [],
// Omited....
}
console.log
insert [Function]
In this case insert
is function and callable.
Can I use transact.insert on Lambda Layer ?
===
async function ScanDynamodbTable(lastKey) {
return await DynamoDB.table('mytable')
.resume(lastKey)
.limit(1)
.scan()
}
Returns a single array object. But if I wanted to resume scanning, it comes from the this.LastEvaluatedKey
which can only be obtained from within scan.
Basically, i want to create a do while loop with async/await, because recursion with promises has a memory leak, and I can't seem to get the LastEvaluatedKey.
any plans to create wrapper for DynamoDB.DocumentClient
?
How would I do it? Does this support that?
Your code looks great.
I'm not sure if you've seen the AWS API Gateway, Lambda implementations, using DynamoDB and node.js.
The way they're doing the update process is painful. My sense is they could use some of your insight.
Can I add your library to Lambda?
Let me know.
Thanks
Joe (Creatively.Me)
Could you please implement backoff and retry capbabilites per http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ErrorHandling.html#APIRetries
If it already exists, could you information about it to the documentation?
I think right now, this module only supports arrays, but I'd like to add unique values only, I think using a String Set is the way that is done.
Can you consider adding that feature.
For now, I modified the add prototype for my testing.
DynamoDB.prototype.addSS = function(data) {
if (Array.isArray(data)) {
var to_ret = {'SS': [] }
for (var i in data) {
to_ret.SS[i] = data[i]
}
return new DynamoDB.Raw({
Action: 'ADD',
Value: to_ret
})
}
}
Do your lib support these functions for set ?
For example: How to pass params to function for these ? I tried to use DynamoDB.SS(['a', 'b']) but get Exception:
[ValidationException: One or more parameter values were invalid: Invalid number of argument(s) for the IN ComparisonOperator]
I'm trying to perform a recursive scan, following the documentation at https://awspilot.github.io/dynamodb-oop/pages/scan/ (tab Nยฐ2)
// continous scan until end of table
(function recursive_call( $lastKey ) {
DynamoDB
.table('messages')
.resume($lastKey)
.scan(function( err, data ) {
// handle error, process data ...
if (this.LastEvaluatedKey === null) {
// reached end, do a callback() maybe
return;
}
var $this = this
setTimeout(function() {
recursive_call($this.LastEvaluatedKey);
},1000);
})
})(null);
Why is there a 1sec timeout? That's huge, what does it do really?
How does this example returns the content of the table? Calls are recursive but I don't see any variable holding the dataset nor updating it. I don't understand how it works.
Could the doc provide a better example with an actual use-case? Like recursive scan a table then display the results, using a reusable function, for instance?
Hello,
I am able to connect to dynamoDB, get list of ducments from database. I could update string values in documents as well. If details is string it is working fine.
DynamoDB
.table('user')
.where('id').eq(customer.id)
.update({
"EmailAddress": customer.EmailAddress,
"data": details
}, function( err, data ) {
console.log("555 "+ err )
console.log( data )
});
But How can I update nested JSON, using this "aws-dynamodb" npm module, when details is nested json.
Please let me know.
Thanks,
Mayuri Zingade
When I stringify my data and pass it into the insert function, the data is inserted with \\
escapes everywhere. Here is my call:
FYI This is fake data.
data = {"data":{"records":[{"id":"caa6b906e3535a5b29d3db2cb5cce6","first_name":"Myhanh","first_pronounced":"","middle_name":"L","last_name":"May","name":"Myhanh L May","language":"English","phone":[{"number":"8123691112","resource_uri":"/v2/records/8123691112/"}],"address":[{"city":"Topeka","extended_zip":"1918","country":"USA","line1":"2510 Se Burton St","line2":"","state":"KS","zip_code":"66605"}],"relatives":[],"email":"[email protected]","line_type":"Landline","carrier":"At&t","linked_emails":["[email protected]","[email protected]"],"social_links":[],"dob":"","age":"45-54","education":"Completed College","gender":"Female","high_net_worth":"","home_owner_status":"Own","household_income":"100k-125k","length_of_residence":"11-15 years","marital_status":"Married","market_value":"150k-200k","occupation":"","presence_of_children":"No","department":"not specified","telco_zip":"66607","telco_zip_4":"1734","resource_uri":"/v2/users/caa6b906e3535a5b29d3db2cb5cce6/"}]}}
DynamoDB
.table('numbers')
.insert({
phoneNumber: number,
data: cleanData //Data gets inserted with tons of escapes
}, function(err,data) {
console.log( err, data );
if(err){
console.log("Error on DB Save");
}
res.statusCode = 200;
res.setHeader('Content-Type', 'application/json');
res.send(response);
next();
});
Here is what a table entry looks like:
{\"data\":{\"records\":[{\"id\":\"e91ed2a024d779ea6b05c2fc54740e\",\"first_name\":\"Dawn\",\"first_pronounced\":\"DAWN\",\"middle_name\":\"\",\"last_name\":\"Rodenbaugh\",\"name\":\"Dawn Rodenbaugh\",\"language\":\"English\",\"phone\":[{\"number\":\"8123691111\",\"resource_uri\":\"/v2/records/8123691111/\"}],\"address\":[{\"city\":\"Holiday\",\"extended_zip\":\"\",\"country\":\"USA\",\"line1\":\"2642 Ochatilla Rd\",\"line2\":\"\",\"state\":\"FL\",\"zip_code\":\"34691\"}],\"relatives\":[],\"email\":\"[email protected]\",\"line_type\":\"Landline\",\"carrier\":\"Verizon\",\"linked_emails\":[\"[email protected]\"],\"social_links\":[],\"dob\":\"\",\"age\":\"\",\"education\":\"\",\"gender\":\"Female\",\"high_net_worth\":\"\",\"home_owner_status\":\"\",\"household_income\":\"\",\"length_of_residence\":\"\",\"marital_status\":\"\",\"market_value\":\"\",\"occupation\":\"\",\"presence_of_children\":\"\",\"department\":\"not specified\",\"telco_zip\":\"33602\",\"telco_zip_4\":\"1702\",\"resource_uri\":\"/v2/users/e91ed2a024d779ea6b05c2fc54740e/\"}]}}
Docs scroll to top when I highlight something in Ace editor and click. New version of Ace fix this issue. I test with Mac+Chrome.
Having a look through the source for routeCall
it looks like this package can't be used with promises (by not passing a callback and calling .promise()
on the AWS SDK service call result). Which is a shame given I love the resulting 'language' this package creates on top of DynamoDB. That callbacks are used internally in dynamodb.js
might make supporting promises and callbacks a bit of a refactor. Are there any other reasons promises aren't supported?
Is there any documentation on how to satisfy/turn off the new schema warning present in latest updates?
โ ๏ธ [AWSPILOT] missing schema for table ...
Thanks!
I wonder if this lib can perform operation such as UpdateTable: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB.html#updateTable-property
I couldn't find such method in the doc http://awspilot.github.io/dynamodb-oop/
Hi,
I'm afraid that new version of your library api is not backward compatible with 0.1.33 version.
For instance in following code
var command = dynamodb.table(tablePrefix + table);
for (var key in query) {
command = command.where(key).eq(query[key]);
}
command.get(callback);
in callback function I used following:
function (result) {
return result[0];
}
after update to version 0.1.44 I had to use return result[0][0] because additional layer in result array appeared.
I using the library and working good. In this point I found this error. I resolved this error using the native aws-sdk and works well.
##Error
{
"message": "Value provided in ExpressionAttributeValues unused in expressions: keys: {:create_at_2_v1, :create_at_1_v1}",
"code": "ValidationException",
"time": "2016-11-23T18:06:22.315Z",
"requestId": "CVAO4IP0SKPNVCULVHTTS7DC8RVV4KQNSO5AEMVJF66Q9ASUAAJG",
"statusCode": 400,
"retryable": false,
"retryDelay": 0
}
##Example
//...
function (init, end) {
const def = new Defer();
const initDate = new Date(init);
const endDate = new Date(end);
console.log(+initDate, +endDate); // log for test
dynamodb
.table('table')
.having('key1').null()
.having('create_at').between(+initDate, +endDate)
.scan(function(err, data) {
if (err) return def.reject(err);
def.resolve(data);
});
return def.promise;
}
//...
Instead of the entire aws-sdk library, we should be able to...
const { DynamoDB } = require("@aws-sdk/client-dynamodb")
const $db = new DynamoDB({ region: 'us-west-2' })
const DynamodbFactory = require('@awspilot/dynamodb')
const dynamodb = new DynamodbFactory($db)
How does this work? I couldn't find an example how to get the consumed capacity to return.
As per documentation, _the recommended use-case for npm-shrinkwrap.json is applications deployed through the publishing process on the registry: for example, daemons and command-line tools intended as global installs or devDependencies.
It's strongly discouraged for library authors to publish this file, since that would prevent end users from having control over transitive dependency updates._
Please remove the file from npm package to prevent such issues in e.g. webpack projects.
The ASIN is also the main key of the table.
update or insert data:
{"ASIN":"B00CIDQ470","date_set_at":"2016-11-23T22:03:53.100Z","lowestPrice":51.59}
object after update:
{ date_set_at: '2016-11-23T22:03:53.100Z', lowestPrice: 51.59 }
I've solved it by cloning the data object and using the clone instead, but I dont think it should be modifying my object.
function update_or_insert_item(key, value, data, callback) {
var tmp_data = _.clone(data) // it's removing ASIN if i dont clone it
console.log('update or insert data: %j', tmp_data)
DynamoDB
.table(dynamoTable)
.where(key).eq(value)
.return(DynamoDB.NONE)
.insert_or_update(tmp_data, function (err, res) {
if (err) {
console.log(err, res)
}
//console.log( 'update result:', err, res )
//console.log(this.ConsumedCapacity)
return callback(err, res)
})
}
Hello
Could you please add this feature
dynamoDb .table("table") .select( ["attribute1", "attribute2", ...] )
Thanks
I have a table feedback
with a rating
field, which is a number.
I'd like to get the average value of this field, is there any simple way of doing it? So far I only see a query
which gets all the rows and do the average calculation once they've all been retrieved.
This would be a big help to my app as Dynamo Transactions opens up a lot of use cases.
Firstly, well done, nice module.
This is not really an issue, just a doco comment that you might consider adding to save someone a bit of time that is new to AWS DynamoDB (maybe why they are using your module).
Basically I was trying to use a value in the data object returned from insert_or_update to do another operation and it took me a while to figure out that the insert_or_replace returned the old changed data!
After reading the AWS doco section on return values I finally understood that DynamoDB returns the old values if attributes change.
http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html
ALL_OLD - If PutItem overwrote an attribute name-value pair, then the content of the old item is returned.
Anyway, I was thinking a comment, perhaps against the console.log message saying something along the lines of, the data object contains the old data if attributes change.
E.g.
DynamoDB
.table('users')
...
.insert_or_update({
...
}, function( err, data ) {
console.log( err, data ); // data object contains the old data if attributes change.
})
Again, cheers for the module.
I've tried using the if function to implement a conditional write on a version property, but it doesn't seem to be working. I see from the implementation you're using the deprecated Expected attribute, are planning to move to the ConditionExpression instead?
I get this weird error:
../node_modules/aws-sdk/lib/request.js:31
throw err;
^
TypeError: Cannot read property 'hasOwnProperty' of undefined
at Object.DynamoUtil.normalizeItem (../node_modules/aws-dynamodb/lib/util.js:103:18)
at Request.<anonymous> (../node_modules/aws-dynamodb/lib/dynamodb.js:477:60)
at Request.<anonymous> (../node_modules/aws-dynamodb/lib/dynamodb.js:279:20)
at Response.<anonymous> (../node_modules/aws-dynamodb/lib/dynamodb.js:274:13)
at Request.<anonymous> (../node_modules/aws-sdk/lib/request.js:355:18)
at Request.callListeners (../node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at Request.emit (../node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (../node_modules/aws-sdk/lib/request.js:668:14)
at Request.transition (../node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (../node_modules/aws-sdk/lib/state_machine.js:14:12)
Code where it happens:
let token = 'x';
let amount = 1;
dynamodb.table('project').where('token').eq(token).index('token').update({
amount: dynamodb.add(amount)
}
No idea why this happens, even updating another column gives this error.
... that this is by far THE BEST EVER DynamoDB wrapper I've worked with. Thanks a million for your efforts.
I know there is still a lot of work to be done on the same, but, the API coverage and the ease of development after using this should be appreciated.
Thanks...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.