blinkdl Goto Github PK

followers: 2.1K following: 1.0 repos: 30.0 gists: 0.0

Name: PENG Bo

Type: User

Bio: RWKV is all you need

Blog: https://www.rwkv.com/

BlinkDL

A minimalist deep learning library in Javascript using WebGL + asm.js. Runs in your browser.

Currently it is a proof-of-concept (inference only). Note: Convolution is buggy when memories overlap.

The WebGL backend is powered by weblas: https://github.com/waylonflinn/weblas.

Example

https://withablink.coding.me/goPolicyNet/ : a weiqi (baduk, go) policy network in AlphaGo style:

const N = 19;
const NN = N * N;
const nFeaturePlane = 8;
const nFilter = 128;

const x = new BlinkArray();
x.Init('weblas');
x.nChannel = nFeaturePlane;
x.data = new Float32Array(nFeaturePlane * NN);
for (var i = 0; i < NN; i++)
    x.data[5 * NN + i] = 1; // set feature plane for empty board

// pre-act residual network with 6 residual blocks
const bak = new Float32Array(nFilter * NN);
x.Convolution(nFilter, 3);
x.CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak);
x.BatchNorm().ReLU().Convolution(1, 1).Softmax();

Usage

<script src='weblas.js' type='text/javascript'></script>
<script src='BlinkDL.js' type='text/javascript'></script>

Todo

PENG Bo's Projects

ai-writer

AI 写小说，生成玄幻和言情网文等等。中文预训练生成模型。采用我的 RWKV 模型，类似 GPT-2 。AI写作。RWKV for Chinese novel generation.

blinkdl

A minimalist deep learning library in Javascript using WebGL + asm.js. Run convolutional neural network in your browser.

blinkdl.github.io

A collection of State of the Art results in AI / ML / DL / RL / CV / NLP.

blinktodo

A minimalist ToDo.txt page. 如果你的ToDo有一百项以上，试试这个基于txt的极简事项管理工具。

bookcnn

《深度卷积网络：原理与实践》现已在淘宝天猫京东当当发售. 这里是其中的代码下载.

chatrwkv

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

fastchat

An open platform for training, serving, and evaluating large languages. Release repo for Vicuna and FastChat-T5.

gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

hua

Hua is an AI image editor with Stable Diffusion (and more).

libai

LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training

linearattentionarena

Here we will test various linear attention designs.

lm-trick-questions

Here we collect trick questions and failed tasks for open source LLMs to improve them.

mingpt-tuned

A *tuned* minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

nala

The Nala markup, to turn a "Natural Language" sentence into a code-like statement. Nala 标注，将自然语言变为编程语言。

pathtracingjs

Path tracing demo with JS in your web browser. 用浏览器JS做路径跟踪渲染。

projectivory

Project Ivory is a simple forum written a few years ago.

rwkv-cuda

The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

blinkdl Goto Github PK

BlinkDL

Example

Usage

Todo

PENG Bo's Projects

Recommend Projects

Recommend Topics

Recommend Org