Giter Club home page Giter Club logo

numpy_neural_network's Introduction

numpy_neuron_network

仅使用numpy从头构建神经网络, 包括如下内容(持续更新中....)

  1. 网络中梯度反向传播公式推导

  2. 层:FC层,卷积层,池化层,Flatten

  3. 激活函数: ReLU、LeakyReLU、PReLU、ELU、SELU

  4. 损失函数:均方差、交叉熵

  5. 模型的保存、部署

  6. 案例学习:线性回归、图像分类

  7. 迁移学习、模型精调

  8. 进阶:RNN、LSTM、GRU、BN

[TOC]

运行工程

环境:python 3.6.x

依赖:numpy>=1.15.0、Cython、jupyter

a) 下载

git clone https://github.com/yizt/numpy_neuron_network

b) 编译nn/clayers.pyx

cd numpy_neuron_network
python setup.py build_ext -i

c) 启动工程,所有的notebook都可以直接运行

jupyter notebook --allow-root --ip 0.0.0.0

基础知识

0_1-全连接层、损失函数的反向传播csdn地址

0_2_1-卷积层的反向传播-单通道、无padding、步长1csdn地址

0_2_2-卷积层的反向传播-多通道、无padding、步长1csdn地址

0_2_3-卷积层的反向传播-多通道、无padding、步长不为1csdn地址

0_2_4-卷积层的反向传播-多通道、有padding、步长不为1csdn地址

0_2_5-池化层的反向传播-MaxPooling、AveragePooling、GlobalAveragePooling、GlobalMaxPoolingcsdn地址

0_3-激活函数的反向传播-ReLU、LeakyReLU、PReLU、ELU、SELUcsdn地址

0_4-优化方法-SGD、AdaGrad、RMSProp、Adadelta、Adamcsdn地址

DNN练习

1_1_1-全连接神经网络做线性回归csdn地址

1_1_2-全连接神经网络做mnist手写数字识别csdn地址

CNN练习

2_1-numpy卷积层实现csdn地址

2_2-numpy池化层实现csdn地址

2_3-numpy-cnn-mnist手写数字识别csdn地址

2_4-对抗神经网络 、csdn地址

经典网络

3_1-VGG

进阶

5-1-RNN反向传播

5-2-LSTM反向传播

5-3-GRU反向传播

5-4-RNN、LSTM、GRU实现

5-5-案例-lstm连续文字识别

6-1-Batch Normalization反向传播

6-2-Batch Normalization实现

numpy_neural_network's People

Contributors

yizt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

numpy_neural_network's Issues

激活函数Tanh的反向传播

您好,最近阅读了您的这个仓库,受益非浅,非常感谢。
但是我对您在nn/activations.py 中的Tanh的反向传播函数有点疑惑。
def tanh_backward(next_dz):
"""
tanh激活反向过程
:param next_dz:
:return:
"""
return 1 - np.square(np.tanh(next_dz))
根据反向传播的求导,我觉得Tanh的反向函数应该是:
def tanh_backward(next_dz, z):
return next_dz*(1 - np.square(np.tanh(z)))
谢谢您的指教。

sigmoid_backward函数是不是写错了

`
def sigmoid_backward(next_dz):

"""

sigmoid激活反向过程

:param next_dz:

:return:

"""

return sigmoid_forward(1-sigmoid_forward(next_dz))

`

sigmod激活函数的反向传播,应该是

dz = next_dz*z_{i+1}*z_{i+1}*W_i^{T}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.