Giter Club home page Giter Club logo

Comments (3)

vasa-develop avatar vasa-develop commented on June 4, 2024 4

Proposed Solution

Introduction

We would use orbit-db-fsstore to start off. orbit-db-fsstore provides a basic but powerful way to manage directories & files (used for referencing), which could be added to AvionDB as an OrbitDB compatible store type.

How the file system would work?

The AvionDB file system object will have a database scope, meaning you will be able to access the file system like this:

const aviondb = await AvionDB.init("DatabaseName", ipfs);

// The file system can be accessed like this
const FileSystem = aviondb.FS(/* file system configuration */)
  • Each aviondb object will have a file system.
  • Each file system will have multiple "buckets". (See below for more details on buckets)
  • Each "bucket" can store multiple files.

Buckets

A "bucket" can be initialized as follows:

const bucket = FileSystem.createBucket({
    name: "BucketName"
    // other possible optons
})

Each bucket can be viewed as a collection of files of some specific context. For eg, we can have a bucket for profile images, and one for song tracks, etc.

Bucket Supported Operations

Following are the operations that could be supported by a bucket object:

// Add a file (This is just an example, not an exact API specification)
// This adds a FileBuffer with FileMetadata to a bucket
bucket.addFile(FileBuffer, FileMetadata, [options])

// Fetch a file (This is just an example, not an exact API specification)
// This returns the FileBuffer for the matching filterCriteria (matched against FileMetadata)
bucket.getFile(filterCriteria, [options])

// Queries/Lists File(s) using file metadata (This is just an example, not an exact API specification)
// Returns the matching FileMetadata(s)
bucket.find(filterCriteria, [options])

// Remove a file (This is just an example, not an exact API specification)
// Removes the FileBuffer & FileMetadata with matching filterCriteria
bucket.remove(filterCriteria, [options])

// Update a file (This is just an example, not an exact API specification)
// Updates the FileMetadata for the matching filterCriteria (matched against FileMetadata)
bucket.updateFile(filterCriteria, FileMetadata, [options])

// Check if a file Exists (This is just an example, not an exact API specification)
// Checks if a file matching the filterCriteria exists (matched against FileMetadata)
bucket.exists(filterCriteria, [options])

// Imports a BucketInstance (This is just an example, not an exact API specification)
bucket.import(BucketInstance, [options])

// Exports an existing BucketInstance (This is just an example, not an exact API specification)
// Checks if a file matching the filterCriteria exists (matched against FileMetadata)
bucket.export([options])

How Bucket works with orbit-db-fsstore

For each AvionDB "database" we have a file system. We can create separate orbit-db-fsstore "directory" for each AvionDB "database" (namespacing using aviondb.address.root as the directory name).

await fsstore.mkdir('/r', 'zdpuAq5ojD6pFdaV2cCXxVCcNvPskThJHhvowvBT7HNmQHn95')
// Creates a new directory /r/zdpuAq5ojD6pFdaV2cCXxVCcNvPskThJHhvowvBT7HNmQHn95

Now, we can create different buckets in within the context of a file system. When we create a bucket named "photos", we create a directory named "photos" inside the above created directory.

await fsstore.mkdir('/r/zdpuAq5ojD6pFdaV2cCXxVCcNvPskThJHhvowvBT7HNmQHn95', 'photos')
// Creates a new directory /r/zdpuAq5ojD6pFdaV2cCXxVCcNvPskThJHhvowvBT7HNmQHn95/photos

Now, we want to add some files to our "photos" bucket. When we run bucket.addFile, we take the FileBuffer and upload that to IPFS node/network and get the CID. Now, we take this CID and the FileMetadata and write this to a file. (This may not be the best approach when it comes to querying the files against FileMetadata. Any better approaches are welcomed)

await fsstore.write('/r/zdpuAq5ojD6pFdaV2cCXxVCcNvPskThJHhvowvBT7HNmQHn95/photos/file1', 
{ 
    data: 'ipfs cid'
    metadata: {/* FileMetadata */}
})

Following this approach, we can remove, update, fetch, find files from a "bucket".

This is a preliminary idea of how we can manage files using AvionDB. All comments, suggestions, concerns are welcomed.

cc @vaultec81 @deanpress @tabcat @angelorc

from aviondb.

vaultec81 avatar vaultec81 commented on June 4, 2024 1

I think this should be implemented at a later date, and be worked on in a separate independent project. Extensive research should be done first.

from aviondb.

angelorc avatar angelorc commented on June 4, 2024 1

Nice proposal. Textile do something similar. I like the bucket approach

from aviondb.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.