Giter Club home page Giter Club logo

image-generation-subnet's Introduction

Stable Diffusion Subnet

Prerequisites Setup

Before proceeding, ensure that Bittensor is installed. Follow the instructions below:

Once bittensor is installed, proceed with the following steps:

1. Install Substrate Dependencies

Start by installing the necessary dependencies to run a Substrate node.

Update your system packages:

sudo apt update 

Install the required libraries and tools:

sudo apt install --assume-yes make build-essential git clang curl libssl-dev llvm libudev-dev protobuf-compiler

2. Install Rust and Cargo

Rust is the programming language used for Substrate development, and Cargo is the Rust package manager.

To install Rust and Cargo:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Update your shell's source to include Cargo's path:

source "$HOME/.cargo/env"

3. Clone the Subtensor Repository

Clone the Subtensor codebase to your local machine:

git clone https://github.com/toilaluan/subtensor.git

4. Setup & Run Subtensor Server

Navigate to the Subtensor directory and initialize the server:

cd subtensor
./scripts/init.sh
./scripts/localnet.sh

NOTE: Monitor the build and initialization outputs. The initial build may take some time, depending on your hardware.

5. Set Up Wallets

Different roles within the subnet, such as subnet owner, subnet validator, and subnet miner, require separate wallets:

  • The owner wallet creates and manages the subnet.
  • The validator and miner must register with the subnet created by the owner to run their respective scripts.

Create a coldkey for the owner role:

btcli wallet new_coldkey --wallet.name owner

Set up the miner's wallets:

btcli wallet new_coldkey --wallet.name miner
btcli wallet new_hotkey --wallet.name miner --wallet.hotkey default

Set up the validator's wallets:

btcli wallet new_coldkey --wallet.name validator
btcli wallet new_hotkey --wallet.name validator --wallet.hotkey default

6. Mint Tokens from Faucet

Tokens are required to initialize the incentive mechanism on the chain and for registering the subnet.

  • The owner needs > τ1000 to create a subnet.
  • Validators & Miners need > τ0 to register with the subnet.

Mint faucet tokens for the owner, validator, and miner as follows:

# For the owner
btcli wallet faucet --wallet.name owner --subtensor.chain_endpoint ws://127.0.0.1:9946 
# Repeat as needed

# For the validator
btcli wallet faucet --wallet.name validator --subtensor.chain_endpoint ws://127.0.0.1:9946 

# For the miner
btcli wallet faucet --wallet.name miner --subtensor.chain_endpoint ws://127.0.0.1:9946 

7. Create a Subnet

Establish a new subnet on the local chain using the following command. The cost is exactly τ1000.000000000 for the first subnet:

btcli subnet create --wallet.name owner --subtensor.chain_endpoint ws://127.0.0.1:9946 

NOTE: The local chain assigns a default netuid of 1 for the first subnet. Subsequent registrations will increase the netuid sequentially. There is a limit of 8 subnets.

8. Register Keys

Register your subnet validator and miner on the subnet to allocate unique slots. The subnet currently supports up to 128 slots.

Commands for registering the subnet miner and validator:

# For the subnet miner
btcli subnet register --wallet.name miner --wallet.hotkey default --subtensor.chain_endpoint ws://127.0.0.1:9946

# For the subnet validator
btcli subnet register --wallet.name validator --wallet.hotkey default --subtensor.chain_endpoint ws://127.0.0.1:9946

11. Add Stake

Bootstrap the incentives on your new subnet by adding stake to its incentive mechanism:

btcli stake add --wallet.name validator --wallet.hotkey default --subtensor.chain_endpoint ws://127.0.0.1:9946

Setting Up the Stable Diffusion Subnet

  • Validators iteratively request prompts from an LLM Server, then send requests to Miners for image generation.
  • Miners generate images and return them.
  • Validators request rewards from the

Rewarding Server and update the weights.

0. Get the Stable Diffusion Subnet

Clone the necessary repositories and install dependencies:

git clone https://github.com/toilaluan/bittensor-fixed-imagenet.git
git submodule update --init --recursive
pip install -r requirements.txt

Install and run Redis:

sudo apt install redis
sudo systemctl restart redis
sudo systemctl status redis

1. Start Prompt & Reward API

Initialize the Prompt and Reward APIs:

# For Prompt API
cd prompt_gen_api
pip install -r requirements.txt
python app.py

# For Reward API
cd reward_api
pip install -r requirements.txt
python app.py

2. Run Subnet Miner and Validator

Define enviroment parameters .env, remember to change url & port based on your system:

PROMPT_ENDPOINT=http://127.0.0.1:15409/prompt_generate
REWARD_ENDPOINT=http://127.0.0.1:15410/verify
MINER_SD_ENDPOINT=http://127.0.0.1:15414/generate
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_LIST=bittensor

Execute the subnet miner and validator, ensuring the correct subnet parameters are specified:

# For the subnet miner
python sd_net/base_miner/miner.py --netuid 1 --subtensor.chain_endpoint ws://127.0.0.1:9946 --wallet.name miner --wallet.hotkey default --logging.debug

# For the subnet validator
python sd_net/validators/validator.py --netuid 1 --subtensor.chain_endpoint ws://127.0.0.1:9946 --wallet.name validator --wallet.hotkey default --logging.debug

Run WebUI: Validator endpoint

We use streamlit for allowing user queue request (prompt) to redis, validator dequeues request from redis then do inference.

streamlit run sd_net/validators/webui.py

image-generation-subnet's People

Contributors

ifrit98 avatar steffencruz avatar rajkaramchedu avatar unconst avatar shibshib avatar toilaluan avatar jameszwifter avatar eugene-hu avatar

Forkers

zenai-vietnam

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.