Comments (6)
Suppose a user's database has over 100mb data. As far as I'm aware, the changeHandler picks up all items available in the database. Does this mean all 100mb will get downloaded on the client side at some point?
Yep, when you call openDatabase, the client loads all items in that database into memory.
Couple things to keep in mind that can help you work around this:
-
A user can create many databases as they want. So both of your suggestions are possible to do on your end (Pagination by date ranges, or number of items). For example, Database X contains January 2019 - March 2019, Database Y contains April 2019 - June 2019, etc. We’ve had thoughts about implementing a library that can smoothly handle more complex logic like this.
-
The client does not load files into memory when you open a database. You need to call getFile to load a file into memory, and you can load byte ranges at a time if you want. So if larger items can be stored as files instead, this would be useful.
We understand that the above still may not be ideal for apps where individual users need to store lots of data that needs to be filtered/sorted/queried. We built Userbase in this way to optimize for end-to-end encryption and simplicity. We felt the current approach we settled on had the best set of tradeoffs, that it would be simplest, both to end developers, and to us implementing, and that it would be useful enough to satisfy the largest number of use cases for traditional web apps.
We also benchmarked Userbase a while back, and found it took ~0.3 seconds to fetch and decrypt 1000 todos in a database, ~0.8 seconds for 10k todos, and ~6.4 seconds for 100k (2.9 GHz Macbook Pro with 100 mb/s network). This is likely inaccurate in Userbase's current form, but this performance (and faster) is certainly possible with our current framework. We would like to spend more time optimizing in the future for sure.
from userbase.
A couple of my users actually do have databases in the 10's of MB. I'll ask and see how it's going for them.
from userbase.
From that user:
"Its way better now, we had [a chat] with over 500 pages and then we did a new one wich is on 223 pages right now. Since we changed and made a new one we dont have much problems anymore, just tiny disconnects sometimes"
Regarding the first one that was having problems:
"... it fails to connect or is still reconnecting after refreshing a few times, that can take a few mins until it works again."
It's possible that this was actually related to an auxiliary service I was using that was unrelated to userbase, though.
from userbase.
@shamblesides @j-berman Thanks a lot for the really helpful responses and helping me understand how things work at the moment. The data load times are pretty fast, so that isn't an issue at all.
I think if necessary, one can implement pagination based on date ranges. It's a good enough solution for me at the moment.
Thanks a lot again!
Feel free to close the issue :)
from userbase.
I could probably provide a somewhat-adequate description of how the database compaction step works but @j-berman would probably do a better job.
from userbase.
"... it fails to connect or is still reconnecting after refreshing a few times, that can take a few mins until it works again."
We do still have some kinks to iron out to prevent this! There is more room for improvement on this front for sure.
from userbase.
Related Issues (20)
- shareDatabase issues
- Feature request: metered trials as opposed to timed trials HOT 3
- forgotPassword with server-side-encryption HOT 1
- Feature Request: 2FA
- How to read a database with admin API? (Question) HOT 4
- How to remove a database? HOT 3
- [userbase-js] passwordHashAlgo is missing in types
- Automatic Tax Collection with Stripe HOT 4
- Close Userbase Node HOT 2
- Prevent registration, or limit app users to 1 HOT 2
- Error "Buffer2 is undefined" when using userbase-js with SvelteKit HOT 3
- Is this project still alive? HOT 1
- sessionLength not being respected, ending early HOT 10
- All users can view all other items created.
- Feature Request: Verify user during signup to avoid dump email ids.
- Service Unavailable for the last couple of days HOT 3
- Missing Data Stored HOT 2
- Third Party Cookie Issues with Stripe
- ServiceUnavailable: Service unavailable. HOT 2
- getItem database API? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from userbase.