Giter Club home page Giter Club logo

tf-on-abci's Introduction

Distributed TensorFlow on ABCI

Login to Interactive Node from Your Local Machine

ターミナルを 2 つ立ち上げて、それぞれ次のコマンドを実行します:

# SSH tunnel
ssh -L 10022:es:22 -l ${ABCI_ACCOUNT} as.abci.ai
# Login to interactive node via SSH tunnel
ssh -p 10022 -l ${ABCI_ACCOUNT} localhost

${ABCI_ACCOUNT} は自身の ABCI アカウントに置き換えてください。

Setup on ABCI

開発環境の構築はインタラクティブノードではなく GPU がある計算ノードで行うことをおすすめします。

Login to Computing Node from Interactive Node

次のコマンドで計算ノードへログインします:

qrsh -l rt_F=1 -g ${GROUP}

${GROUP} は自身の ABCI グループに置き換えてください。

Setup on Computing Node

Load Modules

module load gcc/7.4.0 python/3.8/3.8.2 cuda/10.1/10.1.243 cudnn/7.6/7.6.5 openmpi/2.1.6

Create Virtual Environment

# Create virtual environment
python3 -m venv tf
# Activate virtual environment
source tf/bin/activate

Install Python Modules

# You can install other modules if needed
pip install -U pip
pip install tensorflow
pip install horovod

セットアップが完了したら、次のコマンドで TensorFlow が正しく GPU を認識しているか確認できます:

python -c "from tensorflow.python.client import device_lib; print(device_lib.list_local_devices())"

Run Examples

セットアップが完了したらサンプルのジョブを投げてみましょう。 ジョブは計算ノードからログアウトしてインタラクティブノードから投げます。

次のコマンドでこのリポジトリをクローンします:

git clone https://github.com/sfujiwara/tf-on-abci.git

Examples

サンプルの実行方法は各ディレクトリの README.md を参照してください:

tf-on-abci's People

Contributors

sfujiwara avatar

Watchers

 avatar  avatar

tf-on-abci's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.