Giter Club home page Giter Club logo

clay's Introduction

Discontinuation notice for the Clay Golem

⚠️ Clay Golem (ex. Brass Golem) has been discontinued. The current, completely new incarnation of Golem is called yagna and developed within a separate set of repositories.

Providers

To become a new Golem provider, please follow the instructions in our Provider's handbook

Requestors

On the other hand, if you're interested in running or developing applications on Golem, please have a look at:


Golem

Golem is a decentralized marketplace for computing power. It enables CPUs and GPUs to connect in a peer-to-peer network, enabling both application owners and individual users ("requestors") to rent resources from other users’ ("providers") machines. These resources can be used to complete tasks requiring any amount of computation time and capacity. Nowadays, centralized cloud providers are in control of these services, these platforms are constrained by closed networks, proprietary payment systems, and hard-coded provisioning operations. Golem proposes an open-source, decentralized and user controlled alternative.

One core component to Golem’s built-in feature set is an Ethereum-based transaction system, which enables direct payments between requestors, providers, and software developers.

Learn more with our Docs

Installing and testing

Follow installation guidelines

Usage & troubleshoothing

The most common problems are described in Issues & Troubleshooting section.

Contributing

Please check our Contribution guidelines.

Contact

Help us develop the application by submitting issues and bugs. See instruction here.

You can also send us an email to [email protected] or talk to us on chat.golem.network.

clay's People

Contributors

azawlocki avatar badb avatar banasiakadam60 avatar cameel avatar chfast avatar dybi avatar elfoniok avatar etam avatar gborowik avatar inexxt avatar jiivan avatar kmazurek avatar krigpl avatar ktomala avatar maaktweluit avatar magdasta avatar marmistrz avatar matthewsedam avatar mdtanrikulu avatar mfranciszkiewicz avatar mplebanski avatar nieznanysprawiciel avatar paulperegud avatar prekucki avatar pstiasny avatar shadeofblue avatar tworec avatar weaselix avatar wiezzel avatar wolf480pl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

clay's Issues

Info on installing and running GNR

  • Complete list of dependencies (python packages and other)
  • Ideally on both linux (Ubuntu?) and windows
  • Maybe move from Readme.md to wiki

Ethereum contracts: Bank of Deposits

The version of a code from a nanopayment whitepaper should be put into the repository and subaccounts contract should be written. All contracts must be tested in a Golem Private Network as soon as it will start to work again.
After that updated support for new contracts should be implemented in Golem.

  • Implement SubAccounts contract.
  • Create genesis block for test network
  • Create mining/bootstrap node (aka server).
  • Deploy SubAccounts contract.
  • Implement direct payments.
  • Implement subaccount payments.

Test and describe timeout mechanism

Describe what exactly should happen after timeout for task / subtask has been reached. Description should include both provider and requester scenarios.

Creating temporary files for all platforms

Rendering tasks create sometimes temporary files, for example Blender task is creating "crop file" that describes a rendering window for a subtask. However there is still a problem with creating and naming a file that is platform independant - right now creating a temp file on Ubuntu generates a file named "" instead of random name.

Upnp support

Add support for UPnP to Golem.

Adding this option will eliminate the need to forward ports by hand for some users. Port forwarding is very uncomfortable process or sometimes not even possible for some users. Having it partially automated will significantly improve UX.

User should be able to choose if she/he wants this option or not. There should be additional flag in config file that would turn on/off this functionality.

We should check if there are any python libraries that support that. The most popular one in C (and used in all p2p application) is miniupnp.

Can we replace reosurce network with IPFS (research)?

Assumptions:

  1. Resources that will be used in Golem Worldwide Renderer stands only for files and directories (with right hierarchy).
  2. Resources for different subtask of the same task will often be the same, so the network should be prepared to send same set of resources to all computing nodes.

Facade Python interface for GNR

Extend and improve GNR interfaces:

  • Python interface - a facade class.
  • CLI interface - simple like python gnr-node.py. It should not import any Qt packages.
  • Merge and clean up startup scripts: admmain.py, grandmmain.py, main.py.

User interface design

Describe which functionalities should be available to users and which one are only for developers and testers.

New task type for testing

Add new Golem task type with no external dependencies, to be used for testing. Should be customizable to allow different size of input data, output data and level of computational difficulty.

Show user clear information why the task didn't compute properly

If this information is accessible user should be able to see from gui, why specific subtask failed. Was there problem with verification, timeout or user send error message? Right now we have to look for this information on golem console, log or by checking received files.

Implement accept_resulsts_delay method for tasks

For tasks there should be a added subtask_id and delay_time as an argument. The name should be also changed from "accept" to "set down" or something similar. The result of this method is a delay value, not a boolean as a name is suggesting.

removeTask is it necessary?

It's quite obvious that user should be allowed to cancel computation, ie. paid for previously computed subtasks, but don't accept any other providers issue.
What is not clear is that if there should be special broadcast method indicating that requester doesn't want to compute this task anymore and information about it shouldn't been kept and broadcast further. It may be useful for very long-standing task but generates some additional complexity.

Additional note: we should be sure that when requester decides to cancel task it pays for all remaining computation, ie. results for the subtasks that has already started being computed. Additional question: what should happened when we added redundant computation and some subtasks has been computed by some nodes but not enough to verify them.

Advance verification for LuxRender

Right now there are no implemented methods for advance verificaiton on LuxRender. We canno use "part of a subtask" method from other task types. We probably should check luxcompare software that should does exactly what we require (PSNR check of flm files).

Closing (and killing) Golem cleanup

Describe what should exactly happen when we close Golem node and what should happend when we kill it. Should we write some data to database, kill computation on Virtual Machine, write some states, inform other nodes? Check which elements are implemented and add missing ones.

Trading system prototype

  • First version
    Requestor broadcast his task with maximum price. All providers who can compute task for price lower than defined will be accepted. Final price equals to price offered by provider multiplied by computation time.
  • Second version
    Requestor broadcast his task with maximum price. For some time offers from providers are collected. Finally the set of best providers is chosen. Function for provider validation should be take price, performance and reputation as parameters.
  • Third version:
    Providers may try to cheat and declare too high performance and too high computation time. Therefore requester will try to predict computation time. Benchmark will be run on requester's computer to compute his performance and expected time. Based on this result requester will to predict computation time for each provider based on their performance. Provider will have to accept this time multiplied by declared price as a fixed remuneration for computation. For each subtasks there is defined timeout so there's no risk of computing a subtask forever.

Change timeouts to absolute.

We should control and fix timeout that are used in tasks. Mainly task timeouts and subtask timeouts, but also some other timeouts that may occur in code (eg. in task computer or task keeper for removing old tasks).

There are some additional important problems with that, ie differences with timestamps in different timezone.

Finding a network test platform

How can we test network scenarios (at least simulate many nodes with different IP's)?

We have tried and made some test with imunes, but this topic is still open.

Subtask progress mechanism

How to inform users about subtask progress? How to measure progress?
For Blender i Luxrender task we should probably monitor logs and find specific strings in logs.

Test task directory concurrent access

When user is testing task and it takes a lot of time, he may close test window and try to test another task or test task again. He's not able to this, because test task directory is locked with another thread. Any solution for this problem will be appreciated.

Signing TaskHeader

TaskHeader should have additional field for signature. Otherwise there is a risk that malicious node may produce false task header directed to other node as a requester.

Change client_id in taskmanager to key_id

Previously clients were identified by uuid called client_id. Then we have added identification by public key (called key_id). Task manager and tasks instances still uses client_id (for example to identify if a client has counted a subtask in that task already). This should be replace with a new method.

Golem payments interface + user interface

Interface that should allow user to see past and future payments.

What other methods should this interface make available? Should it allow to make payments before expected time? Should it allow to make see Ethereum account balance? Should it allow to send additional money to Eth account?

Allow multiple nodes to compute same subtask.

Implement redundant verification from: https://github.com/golemfactory/golem-rd/wiki/Verification

First approach (abandoned branch) was here: https://github.com/golemfactory/golem/tree/redundant_computation

One potential approach is described below, but any other rational algorithm will be fine.

Algorithm proposal:
We define replication level x and consensus level c > x/2 and max tries m. Tasks send up to x copies of subtasks to different nodes and wait for the results. If c same answers is received the subtask is verified and all answers that are consistent with the consensus are accepted. If consensus is not reached within x subtasks, than additional m - x tasks may be send. If consensus is not reached subtask is marked as failed.
If consensus is reached before all x subtasks are send, then subtask is accepted.

Necessary changes:

  • subtask verification may be delayed
  • task should have num_subtask parameter
  • total_tasks = m * num_subtasks
  • task may be computed before total task number is reached

Change configuration directory location

Add support for storing node/application data in user defined folder. E.g. --datadir=/tmp/test-node1.

By default it should be standard OS-specific "add data" folder. It can be queried using appdirs package.

Revise Task interface

  • We have both computation_finished() and finished_computation() methods
  • query_extra_data() should probably be changed to a better name
  • Is there a difference between needs_computation() and get_tasks_left() != 0?

Advance verification boundary conditions test

https://github.com/imapp-pl/golem/wiki/Verification

There are two issues for advance singular verification:

  1. Is it always the random test rectangle draw in a proper way? Is it possible that it is too close to the border? What happens if chosen test rectangle size is bigger than a subtask area?
  2. Some renderers settings may produce anomalies on the picture edges, for example one pixel on the edge of the rendered area may be much brighter that it should be. Identifie renderers and settings that produce such anomalies and propose better solutions for those.

Perforamance test task for Blender and Luxtask

Current performance test task was chosen for PBRT renderer (minilight). Maybe we should be choose different task, somehow connected with Blender?
Additionally test task should be connected with environment, not with global config?

Running tasks output should be written to files

Right now tasks output during computation is written to the console. It should be redirected and written to log files. Those log files may also be send back to the requester and be part of verification process.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.