Giter Club home page Giter Club logo

dcbfs's Introduction

Decentralized Cloud-Based File Storage

A way to encrypt and upload files to multiple servers. This project is still in development and not ready for deployment.

This document outlines the structure and philosophy of the project. If you're interested in contributing, please read the contributing guidelines and code of conduct.

Installation

To install dcbfs to /usr/local/bin/, run the following line in a terminal: python setup.py install --install-scripts=/usr/local/bin

How to use

dcbfs is used as a command line tool.

Usage: dcbfs [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  delete    Interface for deleting files
  examine   Interface for examining blocks
  explore   Show uploaded files
  init      Initialize dcbfs, if not already initialized
  retrieve  Interface for retrieving files
  upload    Interface for uploading files

Encryption

AES, CBC. (If you want to argue for different encryption, please open an issue.)

Ledgers

Giant Ledger

Each entry contains id, location, timestamp of each block; and revocation statements.

(Not currently implemented)

Personal Ledger

Each publisher has one personal ledger. Its id is a hash of the publisher's secret, and is encrypted with that secret. It is padded to the standard block size. Every time a publisher uploads a file, their personal ledger block is revoked and a new one is added to the giant ledger (blocks don't do updates; they are only created or deleted). They are either timestamped or the id hash is salted with a nonce (still deciding).

Contains the information necessary (filename, number of blocks, derived key; derived key is hash of filename and key) to retrieve, decrypt, and assemble the blocks that form each original file.

Each file must be stored with a unique name.

Individual Blocks

Each block has padding added to it so they are all the same size. They will have headers which contain the timestamp of when they were last checked and a hash of their id. Blocks are stored on individual machines.

Header: 100 bytes: 64 bytes for a sha512 hash of some identifying information, 4 bytes for a timestamp (unix epoch time), 16 bytes for the initialization vector, 16 bytes for a md5 hash of the content (used as checksum)

Rules for computers participating in the scheme

Giant Ledger Maintenance: Computers look at the ledger to check for the existence of block copies, and see which blocks should be copied or deleted. It checks the recorded locations of each block with a slightly old timestamp and updates the ledger if the block is not present in a location. If a block has too few copies or not enough availability (compute an availability score?), it copies it onto itself (if it has space) with a timestamp and records this on the ledger. If a block has been marked as revoked, and the computer finds it on itself, it deletes it from itself (and records this on the ledger).

Uploading files: Files are encrypted locally, then split up into standard-sized chunks (with padding if necessary), and each block receives a header (see Individual Blocks). The order of the blocks' ids are recorded in another block (that is indistinguishable from file blocks), and the id of that block is recorded in the user's personal ledger (that also looks like a file block).

Deleting files: For a use to delete a file, it issues a revocation statement for all the blocks of the file and the individual file ledger onto the giant ledger.

Local Maintenance: If a computer finds a block on itself with a really old timestamp, it is deleted.

Communication

Uses HTTPS (chosen for anonymous file transfer and ease of setup)

Notes

Uses python 2

dcbfs's People

Contributors

darcymeyer avatar

Stargazers

 avatar Yehan Wasura avatar Ruchika Pande avatar  avatar Jeffrey Shen avatar  avatar Ward Bradt avatar

Watchers

 avatar

Forkers

reichenbachian

dcbfs's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.