Light

reason-wang / mini_lm_lstm Goto Github PK

View Code? Open in Web Editor NEW

1.0 2.0 0.0 14.15 MB

This is a lstm implementation with pytorch. Trained for language modeling.

Python 9.13% Jupyter Notebook 90.87%

lstm pytorch language-modeling

mini_lm_lstm's Introduction

Mini Language Model by LSTM

这是一个微型的语言模型，由PyTorch实现，并由LSTM进行建模，并可以自定义模型深度。

训练

训练可直接运行train.py文件，同时项目也提供了一些参数可供选择，示例代码如下所示：

python train.py --n_step 5 \
                --hidden_size 128 \
                --batch_size 128 \
                --learning_rate 0.0005 \
                --epochs 5 \
                --embed_size 256 \
                --epochs_save 5 \
                --data_dir data/dataset \
                --num_layers 1 \
                --ckpt_dir model/ckpt

正确性验证

项目将实现的LSTM模型与PyTorch库时间的模型输出进行比较，可以通过在命令行中输入以下命令运行并比较结果，如果想修改初始的参数或者输入维度，可以直接修改代码文件。

python model/lstm.py

结果

项目对比了实现的LSTM语言模型的运行结果，具体如下所示，也可以在plot_result.ipynb文件中查看，所有的结果均由手动记录，项目并不提供保存结果的代码。

报告

报告文件在report目录中

mini_lm_lstm's People

Contributors

Stargazers

Watchers

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.