Giter Club home page Giter Club logo

zenbase's Introduction

Zenbase ๐ŸŽ

Welcome to Zenbase, the all encompassing database. It's a secure home for your data that lives in the cloud.

About Zenbase

Zenbase is:

  • Decentralized, peer to peer
  • Fault tolerant
  • Offline first
  • Private
  • Realtime
  • Cross platform. It runs on browsers, and mobile devices. (Servers coming soon!)
  • Auth ready. Decentralized authentication is built in
  • Graph database
  • Persistent
  • Easy to use

HOW???

That was a long list of super awesome features. Basically, Zenbase combines two awesome technologies: GunDB and Sia SkynetDB. GunDB is a Graph database based on WebRTC. Sia SkynetDB is a decentralized Key/Value pair database on a decentralized storage layer.

GunDB offers a flexible database, but lacks decentralized persistence. Centralized relays like S3 or a server are used to persist your data when WebRTC fails.

Sia SkynetDB has decentralized persistence, but lacks most of the features you'd probably want in a database (like relationships and queries).

Zenbase bridges these with a storage adapter for GunDB that persists data to Skynet.

Why Should I Care

  1. It's free / cheaper than you're currently paying for your database. As of November 2020, Sia SkynetDB storage is free. It's unclear when/if that will change but you can expect that it will be cheap if payment is required. Storage on the Sia network currently costs:
  • 3.02 dollars a month per terabyte for storage
  • 0.62 dollars a terabyte for upload bandwidth
  • 1.06 dollars a terabyte for download bandwidth

Source

Even assuming higher storage costs for higher duplication / better web availability, you're looking at cents for a 10gb database. Should payment be required, we intend to host a payment gateway to allow you to pay for your Zenbase easily with your credit card

  1. It's always available. Even the biggest centralized services like AWS have down time. There is good reason to expect that Zenbase won't. Peer devices will each serve as their own database server making your service more available the more people use it. Even if there are no peers, Sia SkynetDB will always be available. There is another 10-30 redundancy on Skynet data, ensuring your data is always close by

  2. It's cross platform. You don't need a server to run it so that's one less thing to set up and one less thing to pay for.

  3. It's realtime and fault tolerant. Need I say more? Pinging your database for updates sucks

  4. It's open source! ๐Ÿ˜

Getting Started (v2.1.x)

Version 2.1.x removes Skynetjs from being bundled with Zenbase. Skynet is providing their own client side bundle. You now need to import both Gun and Skynet before Zenbase. I've also tested it and it works with NodeJS as well

The Interactive tutorial has been updated: https://starboard.gg/nb/nl2QbJ2

<html>
<head>
<script src="https://cdn.jsdelivr.net/npm/gun/gun.js"></script>
<script src="https://cdn.jsdelivr.net/npm/gun/lib/radix.js"></script>
<script src="https://cdn.jsdelivr.net/npm/gun/lib/radisk.js"></script>
<script src="https://cdn.jsdelivr.net/npm/gun/lib/store.js"></script>
<script src="https://cdn.jsdelivr.net/npm/gun/lib/rindexed.js"></script>
<script src="https://skynet-js.hns.siasky.net/4.0-beta/index.js"></script>
<script src="https://cdn.jsdelivr.net/npm/zenbase/src/index.js"></script>
<script>
let hello_world = ""

// Initialize gun.
// [localStorage] We are turning localStorage to false for testing purposes. Generally, you'd want that to be true
// [secret] Secret should be something long and secure. Your data will be saved to Skynet using that secret
// [portal] Skynet portal you'd like to use. Use a portal you trust or run your own. They could potentially manipulate your data (although I don't see why) 
// [debug] Show debug output
// [until] Change the batch time of Gun so that it doesn't attempt to write to storage too quickly
window.gun = new Gun({
    localStorage: false,
    secret: "YOUR_SECRET_HERE",
    portal: "https://siasky.net",
    debug: false,
    until: 2*1000
})

// Put data into gun. This will store in memory, then localStorage (disabled), then Skynet
gun.get('hello').put({ name: "world" });
// Get data into gun. This will pull from memory, then localStorage (disabled), then Skynet
gun.get('hello').on(data => { 
  hello_world = data['name']
	alert("hello: " + hello_world)
})
</script>
</head>
<body>
</body>
</html>

Getting Started (Old)

Interactive tutorial here: https://starboard.gg/nb/nl2QbJ2

<html>
<head>
<script src="https://cdn.jsdelivr.net/npm/gun/gun.js"></script>
<script src="https://cdn.jsdelivr.net/npm/gun/lib/radix.js"></script>
<script src="https://cdn.jsdelivr.net/npm/gun/lib/radisk.js"></script>
<script src="https://cdn.jsdelivr.net/npm/gun/lib/store.js"></script>
<script src="https://cdn.jsdelivr.net/npm/gun/lib/rindexed.js"></script>
<script src="https://unpkg.com/zenbase/dist/main.js"></script>
<script>
let hello_world = ""

// Initialize gun.
// [localStorage] We are turning localStorage to false for testing purposes. Generally, you'd want that to be true
// [secret] Secret should be something long and secure. Your data will be saved to Skynet using that secret
// [portal] Skynet portal you'd like to use. Use a portal you trust or run your own. They could potentially manipulate your data (although I don't see why) 
// [debug] Show debug output
// [until] Change the batch time of Gun so that it doesn't attempt to write to storage too quickly
window.gun = new Gun({
    localStorage: false,
    secret: "YOUR_SECRET_HERE",
    portal: "https://siasky.net",
    debug: false,
    until: 2*1000
})

// Put data into gun. This will store in memory, then localStorage (disabled), then Skynet
gun.get('hello').put({ name: "world" });
// Get data into gun. This will pull from memory, then localStorage (disabled), then Skynet
gun.get('hello').on(data => { 
  hello_world = data['name']
	alert("hello: " + hello_world)
})
</script>
</head>
<body>
</body>
</html>

Donators (In order of contribution)

๐Ÿ† aaronfye
๐Ÿฅˆ griffgreen
๐Ÿฅ‰ bit-7
๐ŸŽ– julietea
๐ŸŽ– slayerizedkoala
๐ŸŽ– prometheusminer
๐ŸŽ– caipeng2006
๐ŸŽ– yaobr
๐ŸŽ– traviagio
๐ŸŽ– luxebeng

... and more

Credits

  • GunDB
  • Sia SkynetDB

More about the project

https://www.indiehackers.com/product/zenbase

zenbase's People

Contributors

fluffy9 avatar pupcakes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

zenbase's Issues

Slow to update value in skynet

Had a discussion on discord about this. I decided to make an issue here to keep track:

It works, but the db is not so reactive - have to refresh the page because data doesn't come to .on() from skynet easily
but eventually it's there

I'm unsure where this problem is coming from. It would either be my code or something more deeply Skynet related. My first guess is that there is latency coming from the Skynet portal. There will be limited workarounds if the issue comes from a portal. The technology is new so I could contact the team and it could just fix itself over time. Otherwise, I could:

  • automatically switch to a backup portal
  • throw an error or something instead of returning stale data

I'll do some debugging and hopefully fix this soon

I looked at your changes! And see bugs!

Hello Fluffy9,

I looked at your changes!
The line:

      if (!data['data']['!']) {cb(null, JSON.stringify(data['data']) || undefined)} else { cb(null, undefined)} })

you just have to change it like this:

cb(null, JSON.parse(data['data']) || undefined) })

The line in "store.put" is much more important for you

       client.db.setJSON(privateKey, key, data).then(() => {cb(null,1)}).catch(err => {

in new:

       client.db.setJSON(privateKey, key, JSON.parse(data)).then(() => {cb(null,1)}).catch(err => {

Now it should be right!!!!!

To test if it works!!!! :
Important for TEST! Please set a new secret!!

  1. just run gun.get('hello').put({ name: "SuperWorld" }); but please before comment out this line:

gun.get('hello').on(data => { hello_world = data['name']; alert("hello: " + hello_world)

Please don't bother if the new SECRET shows a few errors in the browser console! Two passes no longer show errors!

#######

  1. So that it outputs a clean log in the browser console, please repeat the procedure as in number 1!

Now the new value "SuperWorld" is under the key "hello"! all ok
Next, we'll pick up the "hello" key again! see number 3!

#######

  1. only execute gun.get('hello').on(data => { hello_world = data['name']; alert("hello: " + hello_world), but PLEASE first comment out this line gun.get('hello ').put({ name: "SuperWorld" });
    Very important for test nr 3! comment out gun.get('hello').put({ name: "SuperWorld" });

Now with Gun Options (localStorage: false, and radisk: false,) the result "hello: SuperWorld" should be displayed in the browser console!

For gun options, you should use

file: "skydb1",
radisk: false,

Very important! The first option file: "skydb1", is only so that the test does not use the old db from the browser, but creates a new empty one, so that there are no incorrect results!
The second option is also important radisk: false, this tells Gun that should save und get no data in/from the new browser db "skydb1", but with every query that retrieves new data from SKYNET!
So they always have the latest data!

  1. It could be if a friend of yours with the same SECRET in another browser or another location issued the command:
    gun.get('hello').put({ name: "SuperWorld222" });
    To run!

  2. Now if you do nr 3 again!
    The newest result should now show "hello: SuperWorld22" in the browser console!!
    If you didn't have gun options "radisk: false," gun would fetch data from browser db first and then later from skynet and it would be different results!

Have fun testing
Can tell me about the bugs and success

Update packet.json to release-version of skynet-js 4.1.0 and Gun 0.2020.1236 !

Hello Fluffy9,

I noticed that you still use the 3.0.2 version of "skynet-js" in the "package.json", but the version 4.0-beta of "skynet-js" in the readme.md!
On the one hand, this is misleading and causes problems when creating the "webpack" version!

It's recently released release version 4.1.0, and works great here!

Current GunDB version is "0.2020.1236" with a few fixed bugs!

Maybe also change Readme.md to skynet-js 4.1.0?
Here the link:
https://skynet-js.hns.siasky.net/4.1/index.js

########

Furthermore I have two questions:

  1. Is the "dist" directory still necessary?
    If so, then there should also be a current version!

  2. I can also create a "webpack.config.js" incl. "packete.json" modification for webpack?
    Then the end user can create the "dist" version himself!

I already have a new version "zenbase-2.1.8_2-beta-plus" with browser OK and nodejs OK!

Hello Fluffy9,

Yesterday I installed "SkynetLabs/gundb-relay(branch "skynet-nodejs")" on my dev server and fixed errors while doing so
I have now created a super working new version of "zenbase-2.1.8_2-beta-plus" that works both in the browser and in nodejs!
But I think there is still a problem with the gun in the relay!

I haven't published the version "zenbase-2.1.8_2-beta-plus" on github yet because I also use "SkynetLabs/nodejs-skynet" and there in the "src/client.js" the line:

  getJSON: browserClient.db.getJSON.bind(browserClient),

added so that "getJSON" also works in nodejs!
I'm still waiting for the line to be included in the original repo!
Although I also have a version that works without the correction of "SkynetLabs/nodejs-skynet", but the version is extremely big, and double functions for browser and nodejs!

"SkynetLabs/nodejs-skynet" without correction:
// SkyDB
this.db = {
setDataLink: browserClient.db.setDataLink.bind(browserClient),
};

"SkynetLabs/nodejs-skynet" with correction:
// SkyDB
this.db = {
setDataLink: browserClient.db.setDataLink.bind(browserClient),
getJSON: browserClient.db.getJSON.bind(browserClient),
};

If you want to help with the problems then I'll try to upload both versions of my dev server to my githab repo today or tomorrow!

You can contact the developers of "SkynetLabs/nodejs-skynet" to include this line:
getJSON: browserClient.db.getJSON.bind(browserClient),
see above!!!
"db.getJSON" can then also be used in the nodejs version!
Danger! IMPORTANT! "setJSON" is not meant because this doesn't work, only "getJSON"! This is why "getJSON" is compatible with skydb data!

  • How about a cool WebUI?
  • And as an option encryption and decryption of the data by Gun or by skynet (there is no such thing for skydb at the moment!) or
    other secure encryption!
  • Or user DB, via skynet-mysky or Gun SEC?

Help!!!! zenbase-2.1.6 is defect, get NO data, but ERRORS!!!!

Hello Fluffy9,

I tested everything with the original code, even with different browsers, but I always got errors!
But I already found a solution!

########

  1. Here is my test1 (original zenbase-2.1.6 / debug=true) link:
    https://jsfiddle.net/dvhs15wp/1/

Test1 errors:
a) list File '๏ฟฝ' does not have root radix!
b) Error: Expected parameter 'json' to be type 'object', was type 'string', value '{"!":{"":1}}' at throwValidationError (https://skynet-js.hns.siasky.net/4.0-beta/index.js:15552:11) at Object.validateObject
c) Uncaught SyntaxError: redeclaration of var Gun

########
2. Here my Test2 (Original zenbase-2.1.6 / debug=true / without the gun.put command!) Link:
https://jsfiddle.net/dvhs15wp/2/

Test2 errors:
a) list File '๏ฟฝ' does not have root radix!
b) Error: Expected parameter 'json' to be type 'object', was type 'string', value '{"!":{"":1}}' at throwValidationError (https://skynet-js.hns.siasky.net/4.0-beta/index.js:15552:11) at Object.validateObject
c) Uncaught SyntaxError: redeclaration of var Gun

########
!!!Warning!!! If you change the secret, there will be four 404 GET_Errors on the first run, but all further runs are then OK!!!

  1. Here is my Test3 (my patched zenbase-2.1.6-patch1b / debug=true) link:
    https://jsfiddle.net/dvhs15wp/3/

No errors, everything OK and data!

###########
4. Here my Test4 (my patched zenbase-2.1.6-patch1b / debug=true / without the gun.put command!) Link:
https://jsfiddle.net/dvhs15wp/4/

No errors, everything OK and data!

!!!Very important for testing Zenbase!!!
To test if everything is OK, you have to comment out the "gun.put" command and then see if the data can also be loaded without first executing the "gun.put" command!!!

Here is the link to my new zenbase-2.1.6-patch1b :
https://4001kpf6rhpvb1694id2alaoa0kua26bkdspj68jejgelahsdankfno.siasky.net/zenbase/index_2_1_6_patch1b.js

Have fun testing!
With friendly greetings
parajbs

Using skynet on a gun relay peer

Hi! I see you updating the code recently. It's very interesting to try zenbase again (i've experimented with it about a year ago and it was too slow to use zenbase without a relay)

Today I've tried to use zenbase on a node gun relay server. I think it's a nice way to have a no-store free tier exchangeable relays that store and retrieve data from skynet.

Here's my server init script https://github.com/DeFUCC/gun-vue/blob/master/relay/server.js - it's a one-liner gun relay for my https://github.com/defucc/gun-vue project.

It loads and seems to work fine. But after the server restarts it doesn't get previously saved data back. So no permanent storage is achieved. May be I'm doing something wrong?

Can you please help me find the right way to use zenbase with node (v.16, so ES module syntax is OK there).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.