Comments (6)
yeah having that parameter doesn't work is clearly a flaw. Will push that change soon
from dynamo-types.
I am unable to scan record with filterExpresssion. eg : await test.primaryKey.scan({ limit:20 , FilterExpression: ' is_deleted = 0'});
from dynamo-types.
Yes, this is not supported for now for several reasons. But I'll eventually implement this for sure.
from dynamo-types.
This would be a good feature to have. Would you mind elaborate any reasons are you're not supporting this right now (other than, I assume, lack of time)?
from dynamo-types.
Sure. I mean, lack of time IS one of it but not the one i should list here i guess :)
Fundamentally, I'm convinced that avoiding filter-scan expression as much as possible is the best practice of using DynamoDB, and furthermore, you "SHOULD NOT" use filter-scan on a heavy-loaded production system.
As you guys all know, Scan-Filter can be really expensive. if you scan 1 million records to find 1 record that matches the filter, you're paying for "1 million records read", not the "1 record read".
Beside cost-effectiveness, this could break your system entirely by consuming too much-read capacity and DynamoDB auto scalling (assuming you're using it) won't catchup, So all the read operations would fail, or even worse, kill your web server by holding requests too long. (since dynamodb automatically retries those operations that failed by insufficient capacity. So even if it success eventually, it would take much longer)
So while I'm using this library and DynamoDB on massive scale production (meaning > 1k read / write capacity and wide range of fluctuation depends on traffic) I've successfully avoided using Scan-Filter by utilizing Secondary-Index, and honestly I'm sure that was the only way. And, that's the main reason I didn't made this feature. since a) Seemed dangerous, b) I didn't needed.
Also, you can implement this Scan-Filter on your own application layer by filtering records after "scan", and that won't be that much different from just DynamoDB scan filter with some parallelization, and network bandwidth.
But, Obviously, I don't represent every use cases of DynamoDB and i do think there are some cases that Scan-Filter can be handy. That's why i've called it "Eventually supported".
So,
- If any of you guys want to add this, i'm more than welcome to receive PRs and review it :)
- Please share your use-case of Scan-Filter on your projects (especially on "production")! that would be helpful for me to understand the use cases and build API that situation (or not)
@chillenious @singhdeepme
from dynamo-types.
Thanks for your thoughtful response! I actually think it makes a lot of sense to protect users from accidental use. It wouldn't really be a problem in the case I intended it for, but at the same time, even for that it's better to stay safe and follow an alternative route.
Probably good to leave this unimplemented (if people want to, they can use the DynamoDB api directly), but I would suggest to take the FilterExpression out as it currently is confusing to have this in the function signature not backed by an actual implementation.
from dynamo-types.
Related Issues (20)
- AWS_XRAY_CONTEXT_MISSING and AWS_XRAY_DAEMON_ADDRESS is set HOT 4
- batch write ignoring failed writes HOT 2
- Conditional updates? HOT 11
- Feature Request: Ability to specify type in decorator, to allow Class type HOT 2
- Rename the repo 'dynamo-types'? HOT 3
- CreateTable fails when using TTL field with local-dynamo HOT 5
- node version? HOT 5
- Is this project still maintained? HOT 2
- Scan on global secondary index? HOT 3
- Query.GlobalSecondayIndex.scan doesnโt pass IndexName HOT 1
- Add AWS config to documentation HOT 2
- Remove `Decorator` namespace HOT 1
- Table decorator: Create table if it does not exist HOT 1
- Improve documentation HOT 1
- Is there a way to update only part of a map? HOT 2
- Is there a way to apply filters expression on query HOT 1
- no initializer error HOT 2
- Increment value in DynamoDB table HOT 1
- updates on lists
- This repository will no longer be maintained. HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dynamo-types.