carpentries-incubator / machine-learning-trees-python Goto Github PK

What kind of exercises or tasks could we add to Section 7 to make it more interactive?

Note, the markdown for a standard exercise is:

> ## Exercise
> A) Q1
> B) Q2
> > ## Solution
> > A) Q1
> > B) Q2
> {: .solution}
{: .challenge}

It would be helpful to provide a bit more detail about Gini impurity at:
https://carpentries-incubator.github.io/machine-learning-trees-python/02-decision-tree/index.html

What kind of exercises or tasks could we add to Section 2 to make it more interactive?

Note, the markdown for a standard exercise is:

> ## Exercise
> A) Q1
> B) Q2
> > ## Solution
> > A) Q1
> > B) Q2
> {: .solution}
{: .challenge}

The following paragraph (at https://carpentries-incubator.github.io/machine-learning-trees-python/02-decision-tree/index.html) is forward referencing. To understand the paragraph, we need to understand concepts that haven't yet been taught.

While we will only be looking at classification here, regression isn’t too different. After grouping the data (which is essentially what a decision tree does), classification involves assigning all members of the group to the majority class of that group during training. Regression is the same, except you would assign the average value, not the majority.

We should move the discussion of regression to later in the content.

What kind of exercises or tasks could we add to Section 6 to make it more interactive?

Note, the markdown for a standard exercise is:

> ## Exercise
> A) Q1
> B) Q2
> > ## Solution
> > A) Q1
> > B) Q2
> {: .solution}
{: .challenge}

What kind of exercises or tasks could we add to Section 4 to make it more interactive?

Note, the markdown for a standard exercise is:

> ## Exercise
> A) Q1
> B) Q2
> > ## Solution
> > A) Q1
> > B) Q2
> {: .solution}
{: .challenge}

Thoughts after DUSC workshop 15/11/23:

• Using a different dataset that has a more even spread (50:50) between the two classes would make the bias-variance tradeoff clearer
• Also using a non-medical dataset would expand the usability
• Generally needs more programming tasks. If this lesson follows on from the intro to ML course then you can:

1. Set the data pre-processing as a task
2. Get learners to determine accuracy across test/training data from the get go
3. Set a general task at the end to allow user to train on more or on different data types

• The course really could do with highlighting the benefits of random forests and gradient boosting. This can only be done by adding more features sooner.
• Reduce the amount of plotting. It effective early on to visualise decision trees but not effective and time consuming when comparing later models.
• Perhaps ignore gradient boosting entirely. It is skimmed over so fast it doesn't convey any of the benefits or differences over random forests.
• to show the power of random forests try running the model on highly correlated features
• Ideally the code should not be continually renaming the mdl variable, but create new variables for each model to help comparison

I think it's interesting to see how common XGBoost (etc) papers are published using the approach that we've worked through in the lesson. Perhaps add a link to https://pubmed.ncbi.nlm.nih.gov/?term=xgboost+mortality+prediction at https://carpentries-incubator.github.io/machine-learning-trees-python/07-gradient-boosting/index.html

base_estimator -> estimator in recent sklearn

In the random forest page, we specify max_features=1 but the decision boundaries are all bivariate. This makes for a very confusing introduction to random forests
https://carpentries-incubator.github.io/machine-learning-trees-python/06-random-forest/index.html

As highlighted by @alanocallaghan in #26, some of the software used in this lesson has become outdated (in particular sklearn). Bump the sklearn version and update the syntax.

The evaluation/performance section should include discussion of model calibration:
https://carpentries-incubator.github.io/machine-learning-trees-python/08-performance/

Builds are failing with Node.js 16 actions are deprecated.

See: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/

Fix appears to be at: https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runs-for-javascript-actions

What kind of exercises or tasks could we add to Section 8 to make it more interactive?

Note, the markdown for a standard exercise is:

> ## Exercise
> A) Q1
> B) Q2
> > ## Solution
> > A) Q1
> > B) Q2
> {: .solution}
{: .challenge}

Make it clear that the random forest is restricting the variables available at each decision in the node (rather than restricting which variables are available for training of the entire tree): https://carpentries-incubator.github.io/machine-learning-trees-python/06-random-forest/

What kind of exercises or tasks could we add to Section 3 to make it more interactive?

Note, the markdown for a standard exercise is:

> ## Exercise
> A) Q1
> B) Q2
> > ## Solution
> > A) Q1
> > B) Q2
> {: .solution}
{: .challenge}

What kind of exercises or tasks could we add to Section 1 to make it more interactive?

Note, the markdown for a standard exercise is:

> ## Exercise
> A) Q1
> B) Q2
> > ## Solution
> > A) Q1
> > B) Q2
> {: .solution}
{: .challenge}

Consider adding a section on how to explore variable importance for the models (e.g. shapley).

On the performance page, it would be good to add logistic regression as a baseline:
https://carpentries-incubator.github.io/machine-learning-trees-python/08-performance/

What kind of exercises or tasks could we add to Section 5 to make it more interactive?

Note, the markdown for a standard exercise is:

> ## Exercise
> A) Q1
> B) Q2
> > ## Solution
> > A) Q1
> > B) Q2
> {: .solution}
{: .challenge}