dennybritz / nn-from-scratch Goto Github PK

Hey Denny,

I really appreciate the efforts to build this masterpiece of "AI under the hood". Unfortunately I lost my pdf copy of your blogpost and realised that the page seems to be down. Would you mind uploading a pdf version to the repo?

Best regards,
Martin

I checked the gradients you derived against the numerical gradients, and your implementation does not match. It looks like the error is in two places:

1. In calculate_loss, you average the total loss (including the regularization term) over the data batch. The correct implementation should average only the log loss, but not the regularization term.

2. In build_model, the gradients (dW1, dW2, db1, db2) during backprop should be averaged over the data batch. Again, the correct implementation should not include the regularization terms in the average over the data batch.

The following error comes when I try using pip on the given requirements file

Could not find a version that satisfies the requirement functools32==3.2.3.post2 (from -r requirements.txt (line 6)) (from versions: 3.2.3-1, 3.2.3-1, 3.2.3-2, 3.2.3-2)

when we import matplotlib, it gives the error numpy.core.multiarray failed to import.

To solved this I referred this thread and upgraded numpy version to 11.1.

Just thought you should know, if in future somebody has the same problem.

For the build_model() function, I needed to change for i in xrange(0, num_passes): to just range(0, num_passes): to make it work in Python 3.

For the final print statement, I also needed to add parentheses. After those two updates, I was able to get a great result. Thanks!

I tried setup ipython notebook.
I could "pip install -r requirements.txt" after "pip install numpy" and "sudo apt-get install libncurses-dev".
And I could do jupyter notebook.

But when I import sklearn, error occured.
Stack trace is below.
Please help me.

In [1]: import sklearn

RuntimeError Traceback (most recent call last)
RuntimeError: module compiled against API version 0xa but this version of numpy is 0x9

ImportError Traceback (most recent call last)
in ()
----> 1 import sklearn

/home/ubuntu/share/nn-from-scratch/venv/local/lib/python2.7/site-packages/sklearn/init.py in ()
55 else:
56 from . import __check_build
---> 57 from .base import clone
58 __check_build # avoid flakes unused variable error
59

/home/ubuntu/share/nn-from-scratch/venv/local/lib/python2.7/site-packages/sklearn/base.py in ()
8
9 import numpy as np
---> 10 from scipy import sparse
11 from .externals import six
12 from .utils.fixes import signature

/home/ubuntu/share/nn-from-scratch/venv/local/lib/python2.7/site-packages/scipy/sparse/init.py in ()
211
212 from .base import *
--> 213 from .csr import *
214 from .csc import *
215 from .lil import *

/home/ubuntu/share/nn-from-scratch/venv/local/lib/python2.7/site-packages/scipy/sparse/csr.py in ()
11 from scipy._lib.six import xrange
12
---> 13 from ._sparsetools import csr_tocsc, csr_tobsr, csr_count_blocks,
14 get_csr_submatrix, csr_sample_values
15 from .sputils import upcast, isintlike, IndexMixin, issequence, get_index_dtype

ImportError: numpy.core.multiarray failed to import

Thanks for this illustrative example of how to code a simple neural network from scratch. It has been quite useful for introducing students to neural networks. I just stumbled into one little issue: it seems to be that there is a sign mistake in the expression for delta_3 in the Python notebook:
\delta_3 = y - \hat{y}
as far as I can see, this should be:
\delta_3 = \hat{y} - y
It seems to be correct in the code. Do you agree?

Adding the two variable definitions from the web page and putting it just before the calculate_loss function did the trick.

thanks for your great tutorial!

I have an issue when trying to plot "Decision Boundary for hidden layer size 3". Below the error:

ValueError Traceback (most recent call last)
in ()
1 # Plot hasil
----> 2 plot_decision_boundary(lambda x: predict(model, x))
3 plt.title("Decision Boundary for hidden layer size 3")

in plot_decision_boundary(pred_func)
28 # Predict the function value for the whole gid
29 Z = pred_func(np.c_[xx.ravel(), yy.ravel()])
---> 30 Z = Z.reshape(xx.shape)
31 # Plot the contour and training examples
32 plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)

ValueError: cannot reshape array of size 200 into shape (328,461)

Is there any solution for this? Thank you

Hello， Denny Britz！I come from China, a beginner of Deep Learning. Recently, I read your article Implementing a Neural Network from Scratch in Python – An Introduction. I think it is very great and helpful for these beginner like me. So I want to translate it into Chinese and put it online to let more people learn it, and I sincerely want to ask for your agreement.
Look forward to your reply.
(My English is not very good, I hope you can forgive me if somewhere I use wrong vocabulary and grammar.)

Could not find a version that satisfies the requirement functools32==3.2.3.post2 (from -r requirements.txt (line 6)) (from versions: 3.2.3-2, 3.2.3-1, 3.2.3-2, 3.2.3-1)

First, thank you for sharing this project with us!

Could you please add an explicit LICENSE file to the repo so that it's clear
under what terms the content is provided, and under what terms user
contributions are licensed?

Per GitHub docs on licensing:

[...] without a license, the default copyright laws apply, meaning that you
retain all rights to your source code and no one may reproduce, distribute,
or create derivative works from your work. If you're creating an open source
project, we strongly encourage you to include an open source license.

Thanks!