Again in Ch 5, 6, 7 about practical-statistics-for-data-scientists HOT 3 CLOSED

Naive Bayes:
I checked this and I also get these changed values with a fresh Installation of R. As the results were practically identical between the Python and R implementations, I reached out the author of the klaR package to find out if something has changed that would explain this large change. The changelog of the klaR package mentions changes to the Laplace smoothing that may be responsible to this. When I set alpha in Python to about 2000, I get similar results to R now. I wait what the author of the R package responds.

Formatting changes
I'm not going to change the formatting. The main reason is that the same code is also used in the book and I try to keep it as concise/short as possible.

XGBoost / type error line 452
I thought I already added eval_metric='error' to the notebook. Have done it in another notebook but not here. I've done it now. At the same time, I addressed the warning message that the label encoder will be deprecated in xgboost and therefore changed the definition of y to:

X = pd.get_dummies(loan_data[predictors], drop_first=True)
y = pd.Series([1 if o == 'default' else 0 for o in loan_data[outcome]])

That seemed to have also resolved the type error, as I don't see it.

from practical-statistics-for-data-scientists.

Naive Bayes (update):
I fortunately had an older installation of R around, so I could confirm that with an older version, the results are as printed in the book. The old version was R 3.6.1 and new version R 4.0.3. One thing that changed between these two versions is the handling of string columns when reading data files. The difference is in default.stringsAsFactors() which is TRUE in 3.6.1 and FALSE in 4.0.3. When I explicitly set

loan_data <- read.csv(file.path(PSDS_PATH, 'data', 'loan_data.csv.gz'), stringsAsFactors=TRUE)

I get the original results printed in the book.

$posterior
      paid off   default
[1,] 0.3463013 0.6536987

I'm going to change the code to use stringsAsFactors=TRUE

from practical-statistics-for-data-scientists.

Closed with 7b89f15

from practical-statistics-for-data-scientists.