rasbt / python-machine-learning-book-2nd-edition Goto Github PK

The indexing for the discrete convolution example in the figure of page 498 starts with 1 (that is y[1], y[2], y[3]) whereas the example in the text on page 499 (second section of chapter The effect of zero-padding in a convolution starts at index 0 (that is y[0], etc.). If we were to use Python indexing, the index in the figure should be decreased by 1, I guess.

On page 501 for the first case example the text says "n-10, m=5, p=2, s=1 -> ...". I assume the first entry of the line should be "n=10" instead of "n-10".

The image on page 506 doesn't seem to match the text. The image is supposed to display pooling, but instead it shows convolution.

Best regards,
Gary

on this line:

y.shape[0] is already a plain integer.

The logistic regression estimator (initialized on pg:197) of the pipeline failed to converge for the validation_curve object on pg:199 using the model's default solver, which is 'lbfgs', and so the plot was different from the one in the book. When I changed it to 'liblinear', there was no such error and the plot was replicated.
Also, I have not gone through your book completely yet but do you mention about the significance of choosing the right solvers? How important is it? Thank you.

Interested in a http://mybinder.org/ badge/button?

I think all it would take is a requirements.txt or environment.yml that is (automatically) kept in sync with one of your travis build slots. I could also add one of those files just like that but I suspect version drift will soon come and cause breakage. Any ideas on how to keep them synced without too much engineering?

(p225) In the middle of page,
"argmax_i [0.2 \times i_0 + 0.2 \times i_0 + 0.6 \times i_1]" should be "argmax_i [0.2 \times i_0 + 0.2 \times i_0, 0.6 \times i_1]"

Sorry I misunderstood it.
"argmax_i [0.2 \times i_0 + 0.2 \times i_0 + 0.6 \times i_1]" is right. :)

I believe I found a small issue with the nomenclature in chapter 3, section "Maximizing information gain – getting the most bang for your buck" on page 91 of the packt pdf.

It currently reads:

Here, p (i | t ) is the proportion of the samples that belong to class c for a particular node t.

I believe that should be:

Here, p (i | t ) is the proportion of the samples that belong to class i for a particular node t.

(i instead of c).

Hi,
I found your book very useful for a newbie like me and I am trying to recode all your codes...
In the following lines of code:

ppn = Perceptron(eta=0.1, n_iter=10)
ppn.fit(X, y)
plt.plot(range(1, len(ppn.errors_) + 1), ppn.errors_, marker='o')
plt.xlabel('Epochs')
plt.ylabel('Number of updates')

I see that, you use len(pnn.errors_) and there is no problem. But when i try this:

ppn = Perceptron(eta=0.1, n_iter=10)
print(len(ppn.errors_))
ppn.fit(X, y)
plt.plot(range(1, len(ppn.errors_) + 1), ppn.errors_, marker='o')
plt.xlabel('Epochs')
plt.ylabel('Number of updates')

I run into this error: 'Perceptron' object has no attribute 'errors_'. I know something is wrong with
print(len(ppn.errors_)) but i don't know why. Can you explained in more detail? Thank you in advance.

After following chapter 09 deploying to pythonanywhere.com I'm encountering an interesting error when providing feedback after clicking 'correct' or 'incorrect' button. I have followed the book exactly to set up on pythonanywhere.com

Instructions to reproduce:

1. Go to markolonius.pythonanywhere.com and type a review and submit
2. Click correct or incorrect or
3. Receive error:

Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

From error log:

2018-02-16 02:42:02,532: [2018-02-16 02:42:02,528] ERROR in app: Exception on /thanks [POST]
2018-02-16 02:42:02,532: Traceback (most recent call last):
2018-02-16 02:42:02,532:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1988, in wsgi_app
2018-02-16 02:42:02,532:     response = self.full_dispatch_request()
2018-02-16 02:42:02,533:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1641, in full_dispatch_request
2018-02-16 02:42:02,533:     rv = self.handle_user_exception(e)
2018-02-16 02:42:02,533:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1544, in handle_user_exception
2018-02-16 02:42:02,533:     reraise(exc_type, exc_value, tb)
2018-02-16 02:42:02,533:   File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 33, in reraise
2018-02-16 02:42:02,533:     raise value
2018-02-16 02:42:02,533:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1639, in full_dispatch_request
2018-02-16 02:42:02,534:     rv = self.dispatch_request()
2018-02-16 02:42:02,534:   File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1625, in dispatch_request
2018-02-16 02:42:02,534:     return self.view_functions[rule.endpoint](**req.view_args)
2018-02-16 02:42:02,534:   File "/home/markolonius/movieclassifier/app.py", line 72, in feedback
2018-02-16 02:42:02,534:     train(review, y)
2018-02-16 02:42:02,534:   File "/home/markolonius/movieclassifier/app.py", line 29, in train
2018-02-16 02:42:02,534:     clf.partial_fit(X, [y])
2018-02-16 02:42:02,535:   File "/usr/local/lib/python3.5/dist-packages/sklearn/linear_model/stochastic_gradient.py", line 512, in partial_fit
2018-02-16 02:42:02,535:     coef_init=None, intercept_init=None)
2018-02-16 02:42:02,535:   File "/usr/local/lib/python3.5/dist-packages/sklearn/linear_model/stochastic_gradient.py", line 348, in _partial_fit
2018-02-16 02:42:02,535:     self._validate_params()
2018-02-16 02:42:02,535:   File "/usr/local/lib/python3.5/dist-packages/sklearn/linear_model/stochastic_gradient.py", line 95, in _validate_params
2018-02-16 02:42:02,535:     if self.n_iter <= 0:
2018-02-16 02:42:02,536: TypeError: unorderable types: NoneType() <= int()

Error log: https://www.pythonanywhere.com/user/markolonius/files/var/log/markolonius.pythonanywhere.com.error.log

if test_idx:
    X_test, y_test = X[test_idx, :], y[test_idx]
    plt.scatter(X_test[:, 0], X_test[:, 1],
                alpha=1.0, linewidths=1, marker='o',
                s=55, label='test set')

in book, with c='', is not work

• The image on page 366 is supposed to be the pair-wise distance square matrix, but instead it is a repeat of the sample data image from page 365.

• Page 368, 1st paragraph, the text reads:

The first and second columns denote the most
dissimilar members in each cluster, and the third row reports the distance between
those members.

It should say "third column"

• Page 371, last paragraph, the text reads:

As we can see, in this pruned clustering hierarchy, label ID_3 was not assigned to the
same cluster as ID_0 and ID_4, as expected.

It should say "label ID_3 was assigned"

Sincerely,
Gary

In page 340, right after the information gain formula, the sentence should be: "Here x_i is the .."

Assuming that stride is set to s=1 as mentioned in the text, I would have expected the dimension of the valid padding to be 3 x 3, instead it is shown as 2 x 2 in the figure on page 500.

Hi,

I am trying to test the pickled objects, to verify that I can import the vectorizer and unpickle the classifier.

import re
import os
from vectorizer import vect

clf = pickle.load(open(os.path.join('pkl_objects', 'classifier.pkl'), 'rb'))

However, I get this error:

----> 6 clf = pickle.load(open(os.path.join('pkl_objects', 'classifier.pkl'), 'rb'))

AttributeError: Can't get attribute 'tokenizer' on <module 'main'>

What is going on and how can I fix this error?

Thank you!

add readme and mention to

• checkout chapter 1 for setup instructions
• use the requirements.txt file if unsure about software versions
• recommend the jupyter notebook version over the autogenerated script files

plus, add the table of contents.

On pdf page 248 (labeled 221 on the page), there are two equations at the bottom of the page:

The complement of the error should be (1 - epsilon) in the first equation and (1 - 0.25) in the second. They are reversed.

On page 101, you say the RandomForestClassifier uses the entropy criterion, but the code clearly says criterion='gini' both in the text and the downloaded code.

Excellent book, by the way.

There might be a minor typo on page 88. The very last sentence says "... by making decision based on asking ...". I would guess it was supposed to say "... by making decisions based on ... "? Alternatively "... by making a decision based on ..."?

(p286) For the first paragraph, there is no need to delete db file manually because sqlite support DROP TABLE syntax.
c.execute('DROP TABLE IF EXISTS review_db')

(p287) Unfortunately, this add-on is not compatible with recent Firefox. https://sqlitebrowser.org/ is better option.

Under 'Alternative implementations in scikit-learn' section, "The scikit-learn library's Perceptron and LogisticRegression classes ... make use of the LIBLINEAR libraray,..."
As you may know, Perceptron class use BaseSGDClassifier not liblinear. :)

I typed in the same code as in the book and all functions : get_minbatch(),stream_docs(). The code runs correctly but gives 48% accuracy.I also have checked and confirmed that I am using 45000 reviews for traininig and rest 5000 reviews for testing.Please help as I have to further embed it in a web app.

Hello, I tried to use this implementation of MLP on my data but every time at the 4th epoch I get an error when computing term2 in the cost functions, meaning divide by zero encountered in log
How can I get around this problem? Would it be wrong to add an eps like term2 = (1. - y_enc) * (np.log(1. - output+np.spacing(1))) or clipping the output to a min value?
Thank you

The note on page 56 says that the reader will learn about learning curves in Chapter 5, Compressing Data via Dimensionality Reduction. Could it be that Chapter 6, Learning Best Practices for Model Evaluation and Hyperparameter Tuning was meant instead? I may have overlooked them, but I did not see any learning curves in Chapter 5. But they are properly discussed in Chapter 6.

(P186) In note, "to load the Wine dataset from ..." should be "to load the breast cancer dataset from ..."

Going through the book today, I noticed that the code in the book for the Adaline fit method at the bottom of page 37 could not be correct. I checked it against here:

When computing partial derivative of the SSE I think you made and error at indexing the sum. When you change output of the neuron from "z^(i)" to the SUM_i (w_j * x_j^(i)) I think it should be indexed through "j" not "i". As "z" is computed as the sum of products betweem weights and elements in x^(i).

In chapter 3 - Number of misclassified samples with sklear perceptron, there is a small typo in the equation 6/45 = 0.067 -> should be 3/45 (3 misclassified samples).

The book is otherwise excellent!

Hi Sebastian,

On page the first section ends with "... (step 4)". I would believe that should be "(step 3)" for prediction, since the algorithm ends with step 3.

indices = np.argsort(importances[::-1]) # this causes the wrong order for plt.show() at the end of the code chunk (though the values are correct)
indices = importances.argsort()[::-1] # gives correct order

in the remote chance that this was a legit issue, It would be a privilege to be mentioned!

When you don't have internet and you are going to use the file directly from the same directory, for the code to run correctly you have to change the header of the file declaration

df = pd.read_csv('iris.csv', header=None) #Problem

to:

df = pd.read_csv('iris.csv', header='infer') #ProblemFixing

Hi Sebastian!

On page 382, last section, the text says "... defined as the Sum of Squared Errors (SSE) cost function
∇J(w).". I would had expected "J(w)" without the ∇.

On page 136 a sentence says: "By executing the following code, we will now train a
forest of 10,000 trees on the Wine dataset ...". The code uses only 500 trees and later on page 138 (4th row) the text mentions "500 decision trees".

My apologies if I have completely misunderstood this. Regarding feature selection using Random Forest the code on page 138 says :

print('Number of samples that meet this criterion:',
X_selected.shape[0])

I would have expected rather this line of code:

print('Number of features that meet this criterion:',
X_selected.shape[1])

since we are discussing feature selection and we set the threshold criterion for features importances.

Why cannot we rebuild the model by calling tf.train.import_meta_graph() rather than building the whole graph again with build_cnn, as you illustrated in Ch 14, Page 477 Saving and restoring a model in tensorflow. Also , will I be able to give just my saved model to someone without all the fc,conv2d and build_cnn functions code, and would that person be able to just restore the model and do prediction ?
I would be really greatful if you could help with a clear explanation.

Sir, I have one doubt. The MNIST dataset has 784 columns corresponding to the unrolled 784 pixel values.Each pixel(column) has the same range (0-255). I remember you mentioned in the chapter 4 Building good data sets : that feature normalization helps in faster convergence when all features are in the same scale.In MNIST they are already on the same scale but you still normalized them in Chapter 13 where you used keras for MINST prediction. I tried without normalizing pixel values to [-1,1] range and surprisingly the gradient descent didn't converge even after 50 epochs on the same net architecture.

(P134) In 3rd line at the bottom,
"the 10th position in the sbs.subsets_ attribute and.." should be "the 11th position in the sbs.subsets_ attribute and..", because code is sbs.subsets_[10] not sbs.subsets_[9]

Hi Sebastian.

On page 384 in the MLP figure there might be a typo? In the 2nd layer for the last activation unit I would have expected the sub-index to be "d", not "h". That would fit also with the (m x d) dimension of W(h) as mentioned on page 389 (first section).

(p159) In first Eq., \Sigma_i = \frac{1}{n_i} S_W should be \Sigma_i = \frac{1}{n_i} S_i

The images you created for this book are wonderful illustrations of machine learning concepts. What license is applicable to the images? Would it be acceptable to reuse your images in a presentation, blog post, GitHub guide, etc.? If so, how would you like to be attributed?

I see the code is MIT-licensed but wanted to know whether that applies to the other content as well.

The complete code is displayed in a notepad after executing the commands.
This is my folder structure:
flaskapp1/
app.py
templates/
first_app.html
Below are the commands I ran in terminal:
S:>cd flaskapp1
S:\flaskapp1>app.py
error.docx
error.docx

OS: windows10

In the section Dealing with nonlinear relationships using random forests
Unfortunately, there is now a universal approach for dealing with non-randomness in residual plots, and it requires experimentation

I am sort of confused with the above line in the book ,is it "no" instead of now .

(p326) In a paragraph below code block, "Using the 'absolute_loss' as an argument for the residual_metric" should be "Using the 'absolute_loss' as an argument for the loss".

ch12 notebook

##################
# Backpropagation
##################

# [n_samples, n_classlabels]
sigma_out = a_out - y_train_enc[batch_idx]

isn't this supposed to be

sigma_out = np.multiply(-(y_train_enc[batch_idx] - a_out), sigmoid_derivative_out)

Reference: https://github.com/stephencwelch/Neural-Networks-Demystified/blob/master/Part%204%20Backpropagation.ipynb