Giter Club home page Giter Club logo

web3storage's Introduction

web3storage

Project under active deveopment - not production ready


A Python3 package to interact with Web3.Storage.

This package requires the ipfs-car and carbites command line tool (see working with CAR files)

Install ipfs-car command line tool

npm install -g ipfs-car

This tool is needed to creat CAR files behind the scenes.

Install carbites CLI

npm install -g carbites-cli

This tool is needed to split CAR files into smaller chunks for upload

Already Implemented

  • .upload(): Store files using Web3.Storage. (limited to 100MB)
  • .retrieve(): Retrieve a file from Web3.Storage
  • .metadata(): Retrieve metadata about a specific file
  • .http_header(): RetrieveHTTP header information
  • .user_uploads: Lists all previous uploads for the account

Installation

This package is currently only installable via GitHub

pip install -e git+https://github.com/lgloege/web3storage#egg=web3storage

Using the Package

  1. Create an access token by first logging into your account and clicking on your username in the top right corner. Navigate to "Applications" and then "+new token" under "Personal access tokens". Keep this window open while you proceed to step 2 because the token is only displayed once.

  2. Store the token in ~/.web3_storage_token using the folowing command

 { echo 'ACCESS_TOKEN: your_access_token_here' } > ~/.web3_storage_token
  1. start using the ackage
import web3storage

# create a Client object
ws = web3storage.Client()

# upload a file
ws.upload("~/test.file.txt", filename="test-file")

Notes

This project is under active development. Here is a list of improvements:

  • CAR: right this only works with "small" files (<100MB). Need a way to create and upload a CAR
  • improve retrive: retrieve needs some work. I think there is an encoding issue?
  • tests!: need to test uploading and downloading files, not sure how to mock this
  • documentation: need to setup a readthedocs
  • asyncronous functions: use asyncio and aiohttp to write async functions. This could improve speed

web3storage's People

Contributors

lgloege avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

bellyfat

web3storage's Issues

AttributeError

I get an Attribute Error when running the demo code. I have run pip install git+https://github.com/lgloege/web3storage before that, and it installed successfully.

~/web3storage via ๐Ÿ v3.9.10 (web3storage) took 6s 
โฏ /usr/local/bin/python3 /Users/renekuhn/web3storage/web3stor
age.py
Traceback (most recent call last):
  File "/Users/renekuhn/web3storage/web3storage.py", line 1, in <module>
    import web3storage
  File "/Users/renekuhn/web3storage/web3storage.py", line 4, in <module>
    ws = web3storage.Client()
AttributeError: partially initialized module 'web3storage' has no attribute 'Client' (most likely due to a circular import)

Async functions

Is your feature request related to a problem? Please describe.
Not a problem, but async functions should speed up uploading CAR files, since you have to upload many small files

Describe the solution you'd like
Convert all functions with requests to async functions

Can not upload many CAR files

Describe the bug
Does not upload multiple CAR files correctly.

You need split large files into chunks <100MB in size and upload each individually.

To do this efficiently use async functions (see Issue #2 )

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.